Artificial intelligence for livestock: a narrative review of the applications of computer vision systems and large language models for animal farming

AI is already integrated into many daily tasks, including facial recognition and airport security, and its use is expanding into agriculture. AI supports data-driven decision-making in livestock production, particularly in dairy farming. Computer vision systems (CVS) are applied in dairy farming for animal identification, behavior monitoring, feeding intake, body weight estimation, and disease detection. Natural language processing (NLP) and large language models (LLMs) offer potential for optimizing data integration, dimensionality reduction, and knowledge retrieval in agriculture. There is a significant knowledge gap in the use of LLMs in animal farming, presenting opportunities for new research and technological advancements. Artificial intelligence (AI) has become essential for decision-making across various industries. The number of research projects involving AI increased by 2.5 times between 2010 and 2018 compared to the previous decade (Lu, 2019). Companies investing in AI have seen significant improvements in sales, employment, and market value (Babina et al., 2024). This growth explains the increase in AI-related jobs from 2007 to 2018. By 2025, global AI investments are expected to reach 200 billion U.S. dollars (Goldman Sachs, 2023). In agriculture and livestock, Babina et al. (2024) indicated a doubling of positions related to AI between 2015 and 2018 compared to the preceding seven years in the United States. Government initiatives, such as the 2021 investment by USDA-NIFA and NSF of $220 million in 11 new NSF-led AI Research Institutes, support the growth of precision livestock farming (USDA-NIFA, 2021). These investments facilitate sensor-based management, enabling large-scale animal phenotyping beyond traditional methods. Technologies developed through these initiatives can improve automation and decision-making in livestock systems, covering areas such as animal health, management, nutrition, and welfare. There is a great variety of scientifically proven digital technologies currently available and under development for dairy cattle management. Among the technologies, the use of 2-dimensional (2D), 3-dimensional (3D), and thermal cameras combined with computer vision systems (CVS) offers researchers and producers a versatile, less laborious, and contact-free option for high-throughput phenotyping in livestock production (Gunaratnam et al., 2024). There is a plethora of CVS applications for dairy farming, ranging from animal identification to forage yield estimation in pasture systems. However, in this review, we focus on CVS applications that have been more extensively studied in the literature, which include animal identification, body weight (BW) and composition estimation, feed intake and behavior measurement, and lameness detection. Additionally, this review includes a discussion on the use of transformer-based large language models (LLMs) in natural language processing (NLP) and multimodal AI applications and their ability to extract information from text, converting it into contextual embeddings for knowledge retrieval, text feature extraction, text classification, and text summarization (Min et al., 2023). LLMs can facilitate the democratization of knowledge due to their ability to synthesize vast amounts of information, such as the thousands of research papers published in the dairy science field annually (Kononoff, 2024). Building on the increasing importance of AI in agriculture and livestock, this narrative review highlights key AI applications for dairy farming, with a specific focus on current and future applications of CVS and LLMs. This review explores how CVS models enhance decision-making, starting with animal identification and extending to farm-scale efficiency in areas such as BW, health, and welfare. Lastly, the section on LLMs summarizes key concepts about this novel AI application and discusses their current and potential applications in livestock farming. In this section, we discuss the use of CVS for applications related to animal identification, individual feed intake, behavior, health, and welfare. This study explored CVS because of its capacity to capture multiple phenotypes using a single camera. Therefore, studies focusing on 2D and 3D cameras utilizing different sensors, such as red, green, blue (RGB), and infrared, are highlighted. Animal identification allows data to be collected on an individual level and is a crucial step for precision livestock farming. Traditional identification methods, such as ear tags and radio-frequency identification (RFID) systems, are labor-intensive, error-prone, and can impact animal welfare due to ear lesions (Awad, 2016). In contrast, CVS technologies do not require physical contact and have advanced significantly. In CVS, animal identification can be based on biometric features such as muzzle prints, retinal vascular patterns, and coat color (Allen et al., 2008; Li et al., 2017). Despite their high identification accuracy, this review will not further discuss the use of retinal vascular patterns and muzzle prints for animal identification, as their limited range of distance recognition and noise susceptibility might affect their successful adoption in commercial settings (Shojaeipour et al., 2021). Animal identification using distinct coat color patterns, especially in Holstein cows, has been implemented in settings similar to commercial farms (Xiao et al., 2022) with a mean accuracy of over 89% (Zin et al., 2018; Bello et al., 2020; Xiao et al., 2022). Recently, Bergman et al. (2024) developed a system to identify dairy cows based on facial color patterns, achieving an F1 score of 0.928 using 1,230 face images of 77 cows. However, many dairy cattle breeds do not display distinct coat patterns, which makes identifying solid-colored animals challenging for CVS. Thus, researchers proposed the use of body shape for individual identification. For example, Ferreira et al. (2022) achieved an F1 score of 0.80 when combining depth images from the dorsal area and convolutional neural networks (CNNs) to identify individual calves among 38 animals. Despite achieving high accuracies, the use of CVS for animal identification requires large datasets (Xiao et al., 2022) to accurately identify an animal despite changes in appearance, behavior, or lighting conditions. In fact, larger datasets improve a model’s ability to generalize across factors such as lighting conditions, animal posture, occlusion, and camera angles (Das et al., 2024). The number of images needed to train satisfactory models can vary depending on herd size and environmental conditions. Although image labeling for training an AI model is done only once before on-farm implementation, it is labor-intensive and requires specialized labor. These specialized labor needs can limit the implementation of CVS in commercial settings. To address this issue, Ferreira et al. (2023) proposed the use of pseudo-labeling as a semisupervised learning technique. In this method, images that remain unlabeled after prediction are included in the training set, increasing the diversity of examples. This approach led to an observed accuracy increase from 77.5% to 92.7%. To demonstrate the process, a graphical representation of the technique adopted by Ferreira et al. (2023) is presented in Figure 1. Graphical representation of the pseudo-labeling technique for individual identification utilized by (Ferreira et al., 2023). 1st represents the initial step of training with the labeled dataset. 2nd represents the incorporation of pseudo-labels that exceed a given threshold into a new training round. 3rd represents the model retrained with both the labeled data and the pseudo-labels. The bars represent the accuracy of the individual identification model before (bar on the left) and after (bar on the right) pseudo-labeling was adopted. Improving the efficiency of animal identification using CVS requires addressing the challenge of identifying animals with open-set scenarios. Typically, research has concentrated on closed-set scenarios, where test animals have been previously seen in training data from different days. Therefore, the use of closed-set datasets to train CVS can limit their ability to recognize new animals. In contrast, models designed for open-set scenarios are more adaptable and capable of identifying new animals in the herd, being more applicable in real-life scenarios. Andrew et al. (2021) applied the concept of open-set scenarios identification to a dataset of 46 Holstein cows using a Siamese Neural Network (SNN) to identify similarities between animal pairs and create embeddings by The model with of animals achieved accuracy in individual identification. This animals from based on enabling when a new animal is in the herd and information for daily management. this concept to solid-colored et al., learning networks using depth images and which these depth images into for animal identification based on body The utilized of and or similar to Andrew et al. and achieved an accuracy of Although these applications have the potential to be applied to cattle the that to be in scenarios. animal identification commercial farms is a current challenge and needs to be in future Although have been a significant knowledge gap in animal involving both coat color and 3D Ferreira et al. (2022) achieved an F1 score of using over by depth images from the dorsal area with to identify individual calves among animals. However, to studies have animal identification over In scenarios, open-set scenarios and are animals are in and over the years different depending on their and Additionally, such as and lighting both and in settings to be in future animal identification In to animal identification, CVS be for such as and feed Thus, the section of this review on CVS to improve et al., 2020; et al., et al., and et al., et al., feed intake is for the of dairy cows and the of individual feed intake can animal nutrition, management, and Thus, in this section, we studies that utilized CVS to individual feed intake in dairy individual feed intake is and requires specialized such as feed intake systems, which are utilized for research However, current research utilizing CVS has for the future of on-farm individual intake For example, et al. proposed using and depth the feed and to size in dairy cows. The a mean of and when using or and depth sensors, et al. (2022) and depth the feed combined with to size in Holstein cows, achieving an of Recently, et al. (2023) utilized and depth feed combined with an to size in dairy cows and an of Although feed intake estimation in this section, the that the accuracy of their feed intake prediction models was by lighting conditions. future studies using such as in settings. Additionally, individual feed intake by combining animal identification and feed estimation is in Figure in settings such as animals to and might be due to feed being not potential is animals in the are Although these scenarios a challenge for individual feed intake estimation, an to animal behavior the feed Thus, in the section, we the on the use of CVS to animal behavior, a to into the health, and of animals et al., 2021). of that can be feed intake estimation under commercial settings. 1st represents cows the feed 2nd represents animal with and 3rd represents cows from the feed The applications of animal behavior on farms are vast and can have applications with animal nutrition, and In to individual feed intake, CVS can animal such as and For example, et al. using cameras combined with and an accuracy of when and of Holstein cows in et al. (2023) the ability of CVS combined with an to individual feeding behavior of dairy approach of and for the number of mean between and feeding Although are applications for animal behavior using CVS, individual the challenge of identification and in both and open-set scenarios. Additionally, depending on the camera such as a of animals by bars and animals are cameras to the is challenging due to the different For example, et al. (2023) utilized to behavior an F1 score of However, the that the F1 score to when individual identification to their identification the for individual behavior such as and in of the of the In this is an to CVS for a large-scale implementation more Thus, the development of more that are not by environmental or image is is to that animal growth and is with feed intake and Therefore, the section of the utilizing CVS to animal growth and These are for feed and in dairy Animal is a this section explores the application of CVS in BW, and body score these due to their impact on dairy production is a crucial of dairy related to such as and are essential for Additionally, body score can as a for animal and of especially the is a in the and an its the importance of BW, growth is a for development et al., 2021). In this et al. proposed the use of depth cameras to in Holstein cows using features such as in and and a mean of prediction of a to the mean Recently, et al. (2023) applied image processing to extract features of different of the body to in Holstein and cows. The to BW, and their model achieved a of and an of of the observed CVS can facilitate the of features to growth and how body to these such as and is or more because these are not by and accurately growth et al., This information is crucial especially for cows, as animals that are the growth will to growth of yield et al., 2019). Thus, daily growth farms with a to and with and growth that can and application for CVS, especially for cows, is the of is and and changes in body composition can be to through 2D and 3D cameras can be to using CVS. The current on CVS to in dairy cattle is in this section, we a utilizing 2D and 3D For example, et al. 2D cameras to the of and in in dairy cows. The achieved an accuracy of in with a of achieved an accuracy of 3D images to shape and has become more and 3D cameras the of the et al. (2023) on feature images by the between in a 3D and the that the The of and a and from estimation, Although CVS using both 2D and 3D images high accuracy, it is to that the of these models was done by with which is and In studies limited of cows with or to their In fact, et al. a available CVS for in commercial settings and that the system was not to when in cows with or However, these systems utilizing a which is different from the system for et al. (2023) prediction in cows using 3D model achieved accuracy for and a using a on an herd, accuracy from to for and to a et al. (2022) the in and the future use of as a for and studies on using more such as of and shape of different of the initiatives farms by using these to the of related to especially the Although and are utilized for on-farm decision-making, we that can be with the and welfare. Thus, in the section of this review, we discuss a of the studies utilizing CVS to dairy and welfare. and cows is of the and challenging on dairy especially the and animal welfare with and and are of the and that affect dairy health, and welfare. in production, health, and welfare with can be observed in cows across the et al., the of and on animal and Thus, is an to CVS systems to and these In this section, this study explores the current utilizing CVS to dairy and with a focus on and review by et al. (2024) that of the current on lameness is done utilizing CVS. However, a current of CVS in lameness is the of the studies have cameras on and use a of the animal to et al., 2020; et al., et al., 2024). of lameness utilizing is to data by such as and image by cows, or et al., 2023). Thus, is an to camera that of the animals to data and increase the of CVS for lameness in commercial settings of a of the image of a dairy for lameness on commercial settings. et al. (2023) a available CVS lameness system that images to The the CVS across commercial dairy farms and over between the CVS and These represent a application of images for CVS to and identify However, because the a available the key features and models utilized by the system to lameness are which it from being further developed by Therefore, future research focus on the development of lameness models based on images and improve the application of CVS on large of the between and the current on CVS for on the use of to changes in and For example, et al. utilized an camera the of the and a learning to in dairy cows using and proposed system was to with accuracy, and et al. (2022) an camera to the previously system to on both of the The accuracy, and for the of Despite the adoption is limited as the cameras have a limited range and can be by environmental to the use of cameras is to 2D and 3D cameras to changes in and that be with have been with in dairy cows and disease in are by over a of or and can range between to in cows et al., and to for calves et al., under conditions. However, can be labor-intensive and as it is based on In this CVS has been to in dairy cattle production systems. For example, has been utilized to in cows et al., and calves et al., 2019). In to the previously in this review, these studies not have an automation with and on of and through an Recently, et al. (2024) proposed a for the of in dairy cows a using CVS, achieving a of and an of et al. (2023) utilized CVS to the in dairy cows and of and et al. (2023) an of their model using dairy and of and of This the for future research to the adoption of CVS to in calves as a of studies have the herd level for animal However, as the challenge in these is animal identification. For example, the score in is for decision-making, as as the of the and However, individual is crucial for the animal level to improve and welfare. The concept to the herd, disease can affect and their To this section, phenotypes the herd level are to be implemented under conditions. In this future studies focus on applications the individual animal level to improve decision-making by large datasets collected from and This is for addressing current on For example, in the in the United over of face or high of such as data and with technological with new technologies, data limited training a of and applicable et al., 2019). This section explores the potential applications of LLMs in dairy farming. To facilitate of we key concepts in and language we how LLMs can be to extract information from data from various and facilitate dimensionality reduction, data integration, knowledge retrieval, and in the of dairy farming. are currently the LLMs. by et al. designed for their and have their use in such as language et al., computer vision et al., et al., and multimodal representation learning et al., 2021). The technique is based on the to identify of text and how are the text is to a of embeddings a that of text, to their which are a various of importance to the of the text, the of for the training In this review, we focus on proposed applications of as are currently the for many training language the is to the between different in a and in their language models are to prediction et al., 2017). have been proposed for training transformer-based language such as prediction and prediction et al., et al., and learning and et al., 2021). The of transformer-based language which their application in a of and datasets the for a specialized The and language can be in different such as text classification, text and with and training et al., By training transformer-based language models on a large of text, between can be in the of the text, for et al., 2017). Although models from et al., require training to different tasks, of models et al., which billion the can be to similar in various tasks, as by et al., This represents a in AI for the for for training such large models from and enabling to specific using a single model and of that can be by LLMs to a can be a a or to be to the before a For example, a dataset the BW, and current of individual dairy cows be to the as a of training and a is the expected daily production for a Holstein in its with of composition be to on a new the of this review, is a research field with applications being proposed In the of embeddings of of text or which can be by of the neural which in the of et al., 2017). the are training to text into a representation that based on their and For example, the for be to a representation that is to to can represent multiple a by a of the embeddings et al., or by a for text et al., for LLMs are on a large of text, to extract contextual text embeddings from text and the and text into a of which can be as feature for training learning models for This allows data from different which different and to be integrated by converting of data to text and text embeddings that can represent data as a single feature Additionally, text embeddings can be to a larger body of text into a which can be into a of the text or compared with a given text for knowledge retrieval when in a multimodal such as that proposed by et al., text embeddings can be related to image which allows for classification, image based on image and multimodal current challenge in agriculture that might address is data traditional data can be done using with such as and these more of the For this of LLMs such as and can the of However, the challenge is data in agriculture combining data from such as sensors, CVS, and feeding systems, and of which have different and et al., 2020; et al., 2021). In this LLMs can extract features from of and it for future et al., the of a et al., 2020; et al., 2023). In such as et al., and et al., multimodal LLMs have achieved great in datasets for a of However, the potential of using LLMs for data in agriculture and livestock is to be graphical representation of data utilizing text embeddings in dairy cattle production systems is in Figure Graphical representation of the of text embeddings and LLMs to data from different and into features for livestock application of LLMs in agriculture and livestock is to improve the of LLMs the of information from vast of and to et al. (2023) proposed using learning models to extract knowledge from as and from related to cows. the training process, the applied a a and learning before the achieved F1 of and The using the model was due to a of training model proposed applied are with disease The of the proposed by the model was by and achieved a of applications are and can decision-making for animal and In the of can be for that require specific knowledge that might not have been in the training text et al., this a dataset of papers collected from covering related to animal science be as a dataset for knowledge retrieval, and specific be to the by embeddings from the text and with the text embeddings in the dataset. This technique allows the to to the using text in the the and accuracy of such There are current and applications for AI in dairy systems management. This review highlights the research field of CVS for dairy farming. we observed that the current CVS research is focusing on such as the identification of solid-colored open-set for image among the that CVS has the potential to multiple animals using a single in a are in CVS that to be such as data integration, the for in and a knowledge gap the use of LLMs in and livestock systems, the potential for future research to their Despite the for LLMs for their use in agriculture and livestock farming, such as by the models or using research LLMs have the potential to in the dairy related to data and data text LLMs can be further utilized to increase the of information to through knowledge retrieval and text summarization the to information for on-farm is a research in the of Animal and the of and in Animal from the of has developed computer vision systems for in cattle and for of and animal research digital technologies such as and to pasture and animal behavior, with a focus on pasture and in systems. is a research in the of Animal and the of and from the of in precision technologies and for dairy adoption by dairy and to intelligence and data in livestock Ferreira is an and Computer and research in the of Animal and the of in computer science and in dairy science the of in learning and computer vision systems to improve dairy and research is on the of in dairy cows the by data from multiple and computer vision and natural language processing is a the of where in dairy science with applied and in the application of AI to dairy management, particularly on the has global dairy farming has that the gap between data and to the is an the of with the of and the research on intelligence and learning to models for is to animal and as as the production efficiency of livestock systems through these digital The to the support from the of and and This was for by the of Animal The in this are of the and do not the or of the of Animal the or the of The or of Ferreira and