Development of object detection and classification with YOLOv4 for similar and structural deformed fish

Keywords: computer vision, cultured-fish recognition, fish automation, fish classification, YOLO

Abstract

Food scarcity is an issue of concern due to the continued growth of the human population and the threat of global warming and climate change. Increasing food production is expected to meet the challenges of food needs that will continue to increase in the future. Automation is one of the solutions to increase food productivity, including in the aquaculture industry, where fish recognition is essential to support it. This paper presents fish recognition using YOLO version 4 (YOLOv4) on the "Fish-Pak" dataset, which contains six species of identical and structurally damaged fish, both of which are characteristics of fish processed in the aquaculture industry. Data augmentation was generated to meet the validation criteria and improve the data balance between classes. For fish images on a conveyor, flip, rotation, and translation augmentation techniques are appropriate. YOLOv4 was applied to the whole fish body and then combined with several techniques to determine the impact on the accuracy of the results. These techniques include landmarking, subclassing, adding scale data, adding head data, and class elimination. Performance for each model was evaluated with a confusion matrix, and analysis of the impact of the combination of these techniques was also reviewed. From the experimental test results, the accuracy of YOLOv4 for the whole fish body is only 43.01 %. The result rose to 72.65 % with the landmarking technique, then rose to 76.64 % with the subclassing technique, and finally rose to 77.42 % by adding scale data. The accuracy did not improve to 76.47 % by adding head data, and the accuracy rose to 98.75 % with the class elimination technique. The final result was excellent and acceptable

Downloads

Download data is not yet available.

Author Biographies

Ari Kuswantori, King Mongkut’s Institute of Technology Ladkrabang (KMITL)

Department of Instrumentation and Control Engineering

School of Engineering

Taweepol Suesut, King Mongkut’s Institute of Technology Ladkrabang (KMITL)

Department of Instrumentation and Control Engineering

School of Engineering

Worapong Tangsrirat, King Mongkut’s Institute of Technology Ladkrabang (KMITL)

Department of Instrumentation and Control Engineering

School of Engineering

Navaphattra Nunak, King Mongkut’s Institute of Technology Ladkrabang (KMITL)

Department of Food Engineering

School of Engineering

References

Ranney, M. A., Velautham, L. (2021). Climate change cognition and education: Given no silver bullet for denial, diverse information-hunks increase global warming acceptance. Current Opinion in Behavioral Sciences, 42, 139–146. doi: https://doi.org/10.1016/j.cobeha.2021.08.001

Bader, F., Rahimifard, S. (2018). Challenges for industrial robot applications in food manufacturing. ISCSIC '18: Proceedings of the 2nd International Symposium on Computer Science and Intelligent Control, 1–8. doi: https://doi.org/10.1145/3284557.3284723

Goncharuk, A. (2015). Food business and food security challenges in research. Journal of Applied Management and Investments, 4 (4), 223–230. Available at: http://www.jami.org.ua/Papers/JAMI_4_4_223-230.pdf

Dos Santos, A. A., Gonçalves, W. N. (2019). Improving pantanal fish species recognition through taxonomic ranks in convolutional neural networks. Ecological Informatics, 53, 100977. doi: https://doi.org/10.1016/j.ecoinf.2019.100977

Alsmadi, M. K., Almarashdeh, I. (2020). A survey on fish classification techniques. Journal of King Saud University - Computer and Information Sciences. doi: https://doi.org/10.1016/j.jksuci.2020.07.005

Zhao, S., Zhang, S., Liu, J., Wang, H., Zhu, J., Li, D., Zhao, R. (2021). Application of machine learning in intelligent fish aquaculture: A review. Aquaculture, 540, 736724. doi: https://doi.org/10.1016/j.aquaculture.2021.736724

Abinaya, N. S. M., Susan, D., Rakesh Kumar, S. (2021). Naive bayesian fusion based deep learning networks for multisegmented classification of fishes in aquaculture industries. Ecological Informatics, 61, 101248. doi: https://doi.org/10.1016/j.ecoinf.2021.101248

Ahmed, M. S., Aurpa, T. T., Azad, M. A. K. (2021). Fish disease detection using image based machine learning technique in aquaculture. Journal of King Saud University - Computer and Information Sciences. doi: https://doi.org/10.1016/j.jksuci.2021.05.003

Alshdaifat, N. F. F., Talib, A. Z., Osman, M. A. (2020). Improved deep learning framework for fish segmentation in underwater videos. Ecological Informatics, 59, 101121. doi: https://doi.org/10.1016/j.ecoinf.2020.101121

Mohamed, H. E.-D., Fadl, A., Anas, O., Wageeh, Y., ElMasry, N., Nabil, A., Atia, A. (2020). Msr-yolo: Method to enhance fish detection and tracking in fish farms. Procedia Computer Science, 170, 539–546. doi: https://doi.org/10.1016/j.procs.2020.03.123

Salman, A., Maqbool, S., Khan, A. H., Jalal, A., Shafait, F. (2019). Real-time fish detection in complex backgrounds using probabilistic background modelling. Ecological Informatics, 51, 44–51. doi: https://doi.org/10.1016/j.ecoinf.2019.02.011

Jalal, A., Salman, A., Mian, A., Shortis, M., Shafait, F. (2020). Fish detection and species classification in underwater environments using deep learning with temporal information. Ecological Informatics, 57, 101088. doi: https://doi.org/10.1016/j.ecoinf.2020.101088

Fouad, M. M. M., Zawbaa, H. M., El-Bendary, N., Hassanien, A. E. (2013). Automatic nile tilapia fish classification approach using machine learning techniques. 13th International Conference on Hybrid Intelligent Systems (HIS 2013). doi: https://doi.org/10.1109/HIS.2013.6920477

Kutlu, Y., Iscimen, B., Turan, C. (2017). Multi-stage fish classification system using morphometry. Fresenius Environmental Bulletin, 26 (3), 1910–1916. Available at: https://www.researchgate.net/publication/314284234_MULTI-STAGE_FISH_CLASSIFICATION_SYSTEM_USING_MORPHOMETRY

Hu, J., Li, D., Duan, Q., Han, Y., Chen, G., Si, X. (2012). Fish species classification by color, texture and multi-class support vector machine using computer vision. Computers and Electronics in Agriculture, 88, 133–140. doi: https://doi.org/10.1016/j.compag.2012.07.008

Andayani, U., Wijaya, A., Rahmat, R. F., Siregar, B., Syahputra, M. F. (2019). Fish species classification using probabilistic neural network. Journal of Physics: Conference Series, 1235, 012094. doi: https://doi.org/10.1088/1742-6596/1235/1/012094

Mohammadi Lalabadi, H., Sadeghi, M., Mireei, S. A. (2020). Fish freshness categorization from eyes and gills color features using multi-class artificial neural network and support vector machines. Aquacultural Engineering, 90, 102076. doi: https://doi.org/10.1016/j.aquaeng.2020.102076

Pornpanomchai, C., Lurstwut, B., Leerasakultham, P., Kitiyanan, W. (2013). Shape- and texture-based fish image recognition system. Kasetsart Journal - Natural Science, 47 (4), 624–634. Available at: https://www.researchgate.net/publication/289604551_Shape-_and_texture-based_fish_image_recognition_system

Miyazono, T., Saitoh, T. (2017). Fish species recognition based on cnn using annotated image. IT Convergence and Security 2017, 156–163. doi: https://doi.org/10.1007/978-981-10-6451-7_19

Rekha, B. S., Srinivasan, G. N., Reddy, S. K., Kakwani, D., Bhattad, N. (2020). Fish detection and classification using convolutional neural networks. Computational Vision and Bio-Inspired Computing, 1221–1231. doi: https://doi.org/10.1007/978-3-030-37218-7_128

Taheri-Garavand, A., Nasiri, A., Banan, A., Zhang, Y.-D. (2020). Smart deep learning-based approach for non-destructive freshness diagnosis of common carp fish. Journal of Food Engineering, 278, 109930. doi: https://doi.org/10.1016/j.jfoodeng.2020.109930

Villon, S., Mouillot, D., Chaumont, M., Darling, E. S., Subsol, G., Claverie, T., Villéger, S. (2018). A deep learning method for accurate and fast identification of coral reef fishes in underwater images. Ecological Informatics, 48, 238–244. doi: https://doi.org/10.1016/j.ecoinf.2018.09.007

Labao, A. B., Naval, P. C. (2019). Cascaded deep network systems with linked ensemble components for underwater fish detection in the wild. Ecological Informatics, 52, 103–121. doi: https://doi.org/10.1016/j.ecoinf.2019.05.004

Cai, K., Miao, X., Wang, W., Pang, H., Liu, Y., Song, J. (2020). A modified YOLOv3 model for fish detection based on MobileNetv1 as backbone. Aquacultural Engineering, 91, 102117. doi: https://doi.org/10.1016/j.aquaeng.2020.102117

Villon, S., Iovan, C., Mangeas, M., Claverie, T., Mouillot, D., Villéger, S., Vigliola, L. (2021). Automatic underwater fish species classification with limited data using few-shot learning. Ecological Informatics, 63, 101320. doi: https://doi.org/10.1016/j.ecoinf.2021.101320

Ju, Z., Xue, Y. (2020). Fish species recognition using an improved alexnet model. Optik, 223, 165499. doi: https://doi.org/10.1016/j.ijleo.2020.165499

Qin, H., Li, X., Liang, J., Peng, Y., Zhang, C. (2016). DeepFish: Accurate underwater live fish recognition with a deep architecture. Neurocomputing, 187, 49–58. doi: https://doi.org/10.1016/j.neucom.2015.10.122

Islam, M. A., Howlader, M. R., Habiba, U., Faisal, R. H., Rahman, M. M. (2019). Indigenous fish classification of bangladesh using hybrid features with svm classifier. 2019 International Conference on Computer, Communication, Chemical, Materials and Electronic Engineering (IC4ME2). doi: https://doi.org/10.1109/IC4ME247184.2019.9036679

Robotham, H., Castillo, J., Bosch, P., Perez-Kallens, J. (2011). A comparison of multi-class support vector machine and classification tree methods for hydroacoustic classification of fish-schools in Chile. Fisheries Research, 111 (3), 170–176. doi: https://doi.org/10.1016/j.fishres.2011.07.010

Kutlu, Y., Reyhaniye, A. N., Turan, C. (2014). Image analysis methods on fish recognition. 2014 22nd Signal Processing and Communications Applications Conference (SIU). doi: https://doi.org/10.1109/SIU.2014.6830503

Badawi, U., Alsmadi, M. (2014). A general fish classification methodology using meta-heuristic algorithm with back propagation classifier. Journal of Theoretical and Applied Information Technology, 66 (3), 803–812. Available at: http://www.jatit.org/volumes/Vol66No3/18Vol66No3.pdf

Liu, Z., Jia, X., Xu, X. (2019). Study of shrimp recognition methods using smart networks. Computers and Electronics in Agriculture, 165, 104926. doi: https://doi.org/10.1016/j.compag.2019.104926

Pettersen, R., Braa, H., Gaweł, B., Letnes, P., Sæther, K., Aas, L. (2019). Detection and classification of Lepeophterius salmonis (Krøyer, 1837) using underwater hyperspectral imaging. Aquacultural Engineering, 87, 102025. doi: https://doi.org/10.1016/j.aquaeng.2019.102025

Liawatimena, S., Heryadi, Y., Lukas, Trisetyarso, A., Wibowo, A., Abbas, B. S., Barlian, E. (2018). A Fish Classification on Images using Transfer Learning and Matlab. 2018 Indonesian Association for Pattern Recognition International Conference (INAPR), 108–112. doi: https://doi.org/10.1109/INAPR.2018.8627007

Shah, S. Z. H., Rauf, H. T., IkramUllah, M., Khalid, M. S., Farooq, M., Fatima, M., Bukhari, S. A. C. (2019). Fish-pak: Fish species dataset from pakistan for visual features based classification. Data in Brief, 27, 104565. doi: https://doi.org/10.1016/j.dib.2019.104565

Rauf, H. T., Lali, M. I. U., Zahoor, S., Shah, S. Z. H., Rehman, A. U., Bukhari, S. A. C. (2019). Visual features based automated identification of fish species using deep convolutional neural networks. Computers and Electronics in Agriculture, 167, 105075. doi: https://doi.org/10.1016/j.compag.2019.105075

Bochkovskiy, A., Wang, C.-Y., Liao, H.-Y. M. (2020). YOLOv4: Optimal Speed and Accuracy of Object Detection. arXiv. doi: https://doi.org/10.48550/arXiv.2004.10934

Redmon, J., Divvala, S., Girshick, R., Farhadi, A. (2016). You Only Look Once: Unified, Real-Time Object Detection. arXiv. doi: https://doi.org/10.48550/arXiv.1506.02640

Minh, D. H. T., Ienco, D., Gaetano, R., Lalande, N., Ndikumana, E., Osman, F., Maurel, P. (2018). Deep Recurrent Neural Networks for Winter Vegetation Quality Mapping via Multitemporal SAR Sentinel-1. IEEE Geoscience and Remote Sensing Letters, 15 (3), 464–468. doi: https://doi.org/10.1109/LGRS.2018.2794581


👁 42
⬇ 78
Published
2022-03-31
How to Cite
Kuswantori, A., Suesut, T., Tangsrirat, W., & Nunak, N. (2022). Development of object detection and classification with YOLOv4 for similar and structural deformed fish. EUREKA: Physics and Engineering, (2), 154-165. https://doi.org/10.21303/2461-4262.2022.002345
Section
Computer Science