When the infrastructure managers implement an strategy based on corrective maintenance, the results and conclusions derived from track inspection activities allow determining the values of each of its geometry parameters (i.e., vertical profile, alignment, cant, twist and gauge), defining whether there are significant defects or deviations on which it is necessary to intervene (i.e. where maintenance activities need to be carried out on the railway infrastructure). For this reason, inspection operations for the detection of track defects are essential on a regular basis on railway lines in operation.

However, as will be discussed further in the next section, due to the limitations of existing track inspection technologies, these tasks are often very costly, especially if high performance is needed, and involve disrupting or modifying traffic on the line or working during night-time periods.

In addition, the current approach to track inspection and maintenance implies that the limitations of inspection systems are not the only problem, as a paradigm shift is also necessary to move from a corrective approach (acting when the damage is at an advanced stage) to a predictive maintenance strategy, through the implementation of technological solutions that provide information on the prediction of the onset of defects, allowing action to be taken at early stages in the evolution of the deterioration.

This technological breakthrough is vital, as the focus of railway infrastructure managers must be on achieving significant reductions in maintenance costs of increasingly ageing infrastructure while not compromising safety. In fact, the implementation of these new predictive maintenance approaches in the industrial sector has led to cost savings of up to 70% compared to traditional corrective strategies.

Therefore, in order to optimize inspection and maintenance tasks and reduce the high costs associated with them, COREAL’s team has developed an innovative track monitoring system designed to be installed easily and economically on any in-service train vehicle. While in service this train becomes a continuous, real-time register of data related to the condition of the track, that is advanced processed in order to generate predictive maintenance plans, specific and customized to the railway infrastructure on which it is operating.

Our solution to the problem: technology for track geometry inspection based on equipped in-service trains, using inertial methods and mathematical processing

Common methods of track inspection can be grouped into two main categories:

• Manually operated track measuring devices
• Dedicated track inspection trains

Manually operated track measuring devices are cost-effective manually operated tools used to inspect rail and track infrastructure. The most relevant technology in this category is the track geometry measuring trolley.

• Track geometry measuring trolleys consists of a trolley equipped with high-precision sensors capable of measuring all the significant geometry parameters. The position is generally obtained by means of control points with a known position and by measuring distances with a built-in odometer.

  

  As can be seen in the image above, it is a manually operated system (low speeds, need for a specialised technician), which usually has a screen on which the different variables are selected for correct measurement. . Furthermore, trolleys can be combined with GPS or robotised station to store collected data and determine the exact location of measured data.

  Derived from its own nature, the main limitations of this system are its low performance (the equipment circulates at human speed) and the need for the inspection work to be carried out at night or for the railway service to be interrupted.

On the other hand, dedicated track inspection trains are becoming increasingly popular. These vehicles are designed to inspect the track under a circulating load, i.e. the track measuring system consists of a measuring axle which is mounted underneath the vehicle. These inspection vehicles allow numerous analyses and can be grouped into:

• Contact monitoring systems. In this method, a movable roller object, which has constant contact with rails, is used to indicate track geometry parameters. The most common equipment currently used to carry out these tasks are vehicles incorporating contact sensors mounted on track recording vehicles, track tamping-inspection vehicles and draisine-inspection vehicles.

  Thus, vehicles incorporating contact sensors are based, as already mentioned, on physical contact methods, since the sensors are elements placed on the bogies of the inspection trains so that they are in contact with the rails in order to detect defects in the geometry of the track. These contact sensors can be of two types: horizontal sensors (whose function is to measure the inner side of the rails to characterise the alignment and the track gauge) and vertical sensors (whose objective is to measure the running surface to characterise the elevation by taking measurements of longitudinal levelling or profile, cant and twist).

  The main purpose of track tamping machines is to pack the tack ballast. However, some of them can also be levelling-tamping and aligning-tamping machines. These machines, in addition to tamping, measure the defects in the track layout and correct them, placing it in its exact position, grouping these operations in a single machine. The inspection methodology is based on track geometry measuring devices, such as contact sensors.

  A draisine is a self-propelled vehicle fitted with the necessary equipment to carry out maintenance work on the track or catenary, designed to transport operators carrying out maintenance work and the corresponding goods. In addition, it allows the inspection of all geometric parameters as it moves along the track.

  

  One of the main limitations of these systems is that the speed of circulation of this equipment is lower than that of the trains in operation, which makes it necessary to schedule specific periods of time to carry out the work, interrupting the normal operation of the system.

  In addition, by their very nature, the sensors are in continuous contact with the rail at a certain pressure so that they do not separate, which causes continuous wear on the sensors and on the rail, which on the one hand affects the quality of the measurements and on the other the deterioration of the rail.

• Techniques based on non-contact methods. In an attempt to overcome the limitations of contact-based inspection systems, non-contact inspection vehicles were developed in the 1990s. Laboratory cars, which integrate different technologies, are driven by a locomotive or coupled to trains running commercial services.

  They can also be specific self-propelled vehicles.The technologies they incorporate may include such as ultrasonic, thermal sensors or eddy current,  among which those based on optical measurements would stand out. The non-contact optical laser-based measurement system generally operates based on the time of flight principle. To calculate distances and track geometric parameters, the sensor emits a narrow beam light towards the desired object and measures the time taken by the pulse to reflect off the target and return to the device.

  These dedicated vehicles are very sophisticated and expensive, reason why they are limited in number and only can be found in countries with large railway networks. This is the case in Spain, where we find the SENECA laboratory train of Adif for the inspection of the infrastructure of high-speed lines, or the VAI (Installation Inspection Vehicle) of Metro Madrid.

  

  Therefore, we would highlight that their main limitation lies in the fact that they are sophisticated vehicles that are few in number and very expensive, which makes their use very complicated due to the long distance they have to travel to reach from one inspection site to another.
  Another disadvantage, as with tamping machines, is that they occupy the track as they are not commercial train vehicles, which reduces the capacity of the infrastructure, with the consequent added cost.

Given the situation described above, COREAL identified the need to develop an innovative solution capable of solving all the problems described previously, providing results, in terms of track inspection, of equal or superior accuracy to those of the existing systems.
Our technical team focused the strategy on developing a solution based on the concept of non-contact track monitoring vehicle systems, but without having to use dedicated, high-cost track monitoring vehicles.

Thus, the development and validation of an innovative inspection system based on inertial methods and mathematical processing was carried out, which, with the use of sensors located on in-service vehicles, is capable of providing real-time data on the state of the track, on the basis of which predictive maintenance can be implemented.

Our methodology for developing the solution

Once our methodology was implemented, after analysing the technological challenge and the potential client’s objectives, we carried out an research that allowed us to define the best approach to solve the problem, specifying features and assessing their suitability from a cost-benefit point of view.

Thus, we propose to develop an advanced inspection system which, although it consists of a subsystem for capturing and transmitting track records, including the vibrations produced by the different track defects to be measured, its main advantage lies in the mathematical process capable of converting the captured signal to the parameter of interest (track geometry).

This is because the defects are captured indirectly, so the most important technological challenge during the creation of the solution was the development and validation of an advanced mathematical algorithm for data processing, since only through a process of transformation of the digital signal recorded by the inertial sensors as a function of time is it possible to achieve the results and precision that characterizes our system.

With regard to the track acquisition and registration subsystem, the purpose of the inertial sensors is to capture the vibrations produced by the interaction of the vehicle wheel with the track for subsequent analysis. Regarding the location of the sensor, special attention must be paid to signal interference caused by the possible occurrence of resonance phenomena of the system, which is why accelerometers should be installed in those parts of the vehicle with higher eigenfrequencies, such as unsprung masses.

In addition, these sensors must be connected to the data acquisition system by wiring, so their location must be such that the cables can be routed inside the vehicle without excessive difficulty, so as to avoid damage due to the relative movement between the different vehicle masses. For all these reasons, it is determined that the ideal place to install the accelerometers is the vehicle’s axle box. The location of the accelerometers on the axle box of a train is shown below:

As shown in the figure above, the sensor is installed on the metal structure by means of a magnetised joint reinforced with an industrial adhesive. This allows the sensor to be positioned without the need to modify the vehicle’s elements or perform mechanical operations on it, as well as allowing quick and easy assembly/disassembly.

The sensor is connected to the recording unit by means of cables which are anchored to the vehicle’s components by means of cable ties, thus ensuring that they cannot come into contact with moving parts. In addition, the cable is provided with an insulating protective barrier to prevent wear or breakage caused by the projection of stones or other particles.

On the other hand, a GPS geolocation system is implemented which, as acceleration records are made, associates global coordinates of the exact place where these records have been made. The GPS module together with its antenna shall be installed in the vehicle cab. The installation of the antenna can be carried out by magnetic connection with the body of the vehicle or by fastening it by means of cable ties to some element of the vehicle.

Once the data have been collected, they are analysed using the signal processing algorithm. The aim of this processing is to characterise the parameters of the track and its elements from the vibration signal recorded by the inspection system placed in the vehicle, and to be able to relate this information by associating these records with the GPS coordinates.

The algorithm will process the acceleration (vibration) data obtained in the time domain. By means of the acceleration, the displacements that have produced it can be obtained via integration and, by filtering and transforming them to the frequency domain, the signal can be correlated with the cause by means of an analysis based on a mathematical model of the interaction of the vehicle’s axle with the track.

Once this process has been completed, it is possible to return to the time domain to finally obtain in the space domain the correlation of the defects and characteristics of the track with the coordinates of the points where they have been recorded.

The results of this analysis are presented to the client via a web platform, where all track geometry track parameters are displayed, indicating the condition of the track. In this way, it will be able to quantify and predict the moment at which degradation is going to occur, i.e., it will make it possible to anticipate as far as possible, preventing maintenance tasks from being carried out at advanced stages of deterioration.

Conclusions

Our team has developed a system with the capacity to characterise the track condition that can be integrated into any in-service vehicle, so that it is able to monitor all the geometry parameters as the vehicle runs during its normal operation.

Our innovative solution makes it possible to monitor the condition of the network and implement predictive maintenance techniques without affecting track service, so that railway infrastructure managers have a tool that makes it possible to optimise maintenance operations while improving the safety, quality and service level of their infrastructures.

For more information on this innovative solution, do not hesitate to download our specific brochure.

In addition, if you are interested in improving the productivity of your industrial sector, do not hesitate to check out other solutions developed by our team or contact us to jointly develop a tailor-made technological solution to meet your specific needs.

References

Falamarzi, A., Moridpour, S., & Nazem, M. (2019). A review on existing sensors and devices for inspecting railway infrastructure. Jurnal Kejuruteraan, 31(1), 1-10.

Real, J., Salvador, P., Montalbán, L., & Bueno, M. (2011). Determination of rail vertical profile through inertial methods. Proceedings of the Institution of Mechanical Engineers, Part F: Journal of Rail and Rapid Transit, 225(1), 14-23.7

Real Herráiz, J. I., Montalbán Domingo, M. L., Real, T., & Puig, V. (2012). Development of a system to obtain vertical track geometry measuring axle-box accelerations from in-service trains. Journal of Vibroengineering, 14(2), 813-826.

Real, T., Montrós Monje, J., Montalbán Domingo, M. L., Zamorano, C., & Real Herráiz, J. I. (2014). Design and validation of a railway inspection system to detect lateral track geometry defects based on axle-box accelerations registered from in-service trains. Journal of Vibroengineering, 16(1), 234-248.

The first year of production takes place in controlled freshwater environments (hatcheries), where the alevin are born and develop into young salmon (parr), weighing approximately 100-250 gr. and measuring between 12-15 cm. After smoltification (a process of adaptation to salt water) they are taken to raft cages at sea, where they begin to be fed special fattening diets. After two years, when the smolt salmon have reached a weight of between 4-6 kg, they are either harvested or, in the case of the brood fish, returned to the hatcheries for spawning.

The phase when production is most vulnerable is undoubtedly the period during which the salmon remain at sea, as they are exposed to conditions that are out of control, such as weather or water quality. In this sense, the problem began when the salmon industry experienced a major crisis in 2007-2008 following the appearance of the ISA (Infectious Salmon Anemia) virus, which led to a significant drop in production to around 60%. Today, the ISA virus still exists, in fact, outbreaks have emerged (June 2015).  In response, farmers have been forced to increase their use of antibiotics.

Being able to reduce the use of antibiotics would not only mean considerable economic savings but the salmon obtained would be of much higher quality, thus increasing the selling price and profits for the fish farm.

Our solution to the problem: gender classification of salmon at early ages

This increase in antibiotic use could be reduced if, during the fattening stage carried out at sea, farming were carried out differently depending on the gender of the salmon, as it is the males that are more prone to the development of diseases, the main reason for this increase in antibiotic consumption.

In this regard, it should be noted that the mixing of male and female salmon during their breeding produces the early maturation of the former (which usually leads to a stagnation in weight gain), so separating male and female specimens would cause an increase in the size of both genders (at least 9.9% and 11.8%, females and males respectively) by avoiding early sexual maturity and a decrease in diseases in both sexes due to transmission from mating.

In addition, from the point of view of raising alevin, it would also be advantageous to have a higher proportion of females in the facility’s broodstock because, by being able to fertilise the eggs (salmon roe) of more than one female with the sperm produced by a male, a greater number of fertilised roe will be achieved.

However, the analysis and gender classification of salmon at juvenile stages is a challenge for the industry today. The methods currently used in Chile are based on the use of ultrasound equipment with which an expert manually identify the gender of salmons from 400-1,000 grams in weight, not being able to identify them at earlier ages.

Ultrasonography (ultrasound) is a diagnostic method by images based on the use of sound waves and the subsequent capture of the echo that is produced when it bounces off the different organs and tissues, which is why it could constitute an optimal starting tool for our objectives since this technology is used in the study of organs and soft parts, organ measurements, fetal movements, etc.

Nonetheless, while it can be used to diagnose gonadal maturity and gestation stages in females, it does not allow for accurate gender classification of juvenile salmon (parr), as it is difficult to identify their gonads with the naked eye when the degree of maturity of the salmon is so low. In addition, these aquaculture services are very labour-intensive as they are currently only carried out manually.

In view of the shortcomings and limitations of current means of determining and classifying the gender of salmon, the opportunity was identified to develop and validate a rapid, automatic, cheap and reliable method for correctly identifying the gender of salmon at earlier ages, with the aim of achieving differentiated production between males and females when they are taken to sea, with all the benefits that this would imply.

Our methodology for developing the solution

Once our methodology was implemented, after analysing the technological challenge and the client’s objectives, we carried out a research that allowed us to define the best approach to solve the problem, specifying performances and carrying out a cost-benefit analysis.

In this sense, after an analysis of the state of the art, we determined that the use of ultrasound scans as a starting point was appropriate, but in order to achieve an automatic and intelligent salmon gonadal diagnosis and differentiation system of high precision and low cost, which would make possible the sexual identification of the salmon at very early ages (12-15 cm long, weighing between 100-200 gr), just before being taken to the salt water cages, it was necessary to advance through research and development activities.

Thus, it was necessary to develop elements of counting, gender classification and automatic separation that carry out the automatic analysis of each of the fish through the mathematical treatment of ultrasonic images, without the need for these to be interpreted by any expert.

In order to do this, it was first necessary to develop a specific image segmentation algorithm because, although the images taken with the commercial ultrasound machine allow the raw images to be obtained, i.e. the graphic representation of the ultrasound scans in grey scale, these generated images present interferences (noise) and certain anomalies that could hinder the subsequent work of the classifier. For this reason, in order to achieve this first challenge, it was necessary to put into practice a series of pre-processing techniques in order to improve the quality of the image as much as possible and to highlight only those objects that were desired to be identified over other bodies that appeared in the image, but did not correspond to objects of interest (the latter were known as artefacts). In other words, an attempt was made to apply an initial filtering process known as binarization process, the result of which was a black and white image instead of a grayscale image.

In addition, a further step in image processing was required and the study was focused on those areas of potential interest (abdominal area of the fish) so that the smallest possible image cutout could be made around the area containing the objects to be detected (in this case the gonads). In other words, we tried to apply a second filtering known as segmentation with the aim of reducing false positives during the subsequent processing of the classifier and minimizing the computational cost of data processing.

  Diagram of the binarization process, first step within the pre-processed module

Once this first objective was achieved, the second challenge was to design and develop the fish gonad analysis module based on a binarized and segmented image of the areas potentially susceptible of hosting the salmon stomach and gonads.

Thus, with the completion of the work linked to the first challenge, a binarized and segmented image of the areas potentially susceptible of presenting the salmon’s stomach and gonads was available. This reduced the size of the image to be processed and eliminated numerous artifacts that would make it difficult to classify the salmon by gender, but this image was still too large to analyse the existence of reproductive organs. For this same reason, a process based on Histograms of Oriented Gradients was required to be applied in the vicinity of the stomach walls until the salmon’s reproductive organs were located.

Histograms of Oriented Gradients (HOG) are based on the idea that objects (such as salmon gonads) can be characterized by their appearance. To do this, Histograms of Oriented Gradients obtain the orientation of the gradient of each pixel, thus generating the spatial distribution of the object.

In general terms, the main idea that was pursued was to implement a system that divided the image into small regions (known as cells) and obtained a histogram for each of them from the orientation of the gradients of the pixels that form it. In addition, for a better response, the contrast had to be normalized in larger areas (called blocks) and this result had to be used to normalize the cells of the block.

  Stages of the HOG descriptor.

In this way, the combination of the histograms generated for each of the cells will provide the representation of the image in the characteristics space. A Bayesian classifier analyses the regions of the ultrasonic image to characterise the pixels and determine the gonads.

A key premise was made for the final salmon gender determination: If reproductive organs are located within the gonads, given the early age of the specimen, these will be female reproductive organs (as male ones take longer to develop).

Like so, if the Bayesian classifier detects that gonads exist in at least 4 of the subdivisions it will be considered a positive result. This is done to avoid false positives in any occasional subdivision because of a structure similar to them.

Based on this premise, below are two ultrasound images of a female salmon and a male salmon, correctly classified by the algorithm and where the presence of gonads is visually appreciated in the case of the female (in green), not discerned in the case of the male (in red):

Ultrasound scan of female salmon (left) and male salmon (right)

Once analysed and classified, a mechanical valve separator takes each of the juvenile salmon (parr) to one or other of the rafts depending on their gender.

Automatic system separator based on the gender of the salmon

Finally, and as the last step of our methodology, this prototype was validated and tested through a series of tests carried out in order to verify the correct functioning of the system.

The measurement campaign took place in two freshwater aquaculture farms in order to increase the representativeness of the results. Specifically, Coho Salmon and Salar Salmon ultrasound scans were taken at the facilities of the company Salmones Austral S.P.A. in Lake Rupanco. These facilities are located, respectively, in the BioBio Region and the Los Lagos Region.

Fish bank from which samples were taken for validation of the new system

Conclusions

Through mathematical image processing we are able to automatically determine the existence of gonads in salmon at very early ages (12-15 cm long, weighing between 100-200 gr), something that is not possible through the manual classification methods based on ultrasound scans currently used.

In this case, Artificial Intelligence makes possible a process that allows to increase productivity and safety in the aquaculture industry while guaranteeing a higher price, and quality product for the consumer.

To find out more about this innovative solution, please download our specific brochure.

In addition, if you are looking to improve the productivity of your fish farm, do not hesitate to contact us to jointly develop a tailor-made technological solution that responds to your specific requirements.

References

AQUA – Aquaculture and fishing. Salmonid crops in Chile increased by 8.7% in 2017′. (2018)

Ban, M., Hirasawa, K., & Ezure, M. (2008). The Effects of Growth on Sexual Maturation in Sockeye Salmon. Aquaculture Science, 56(4), 605-606.

Davidson, J., May, T., Good, C., Waldrop, T., Kenney, B., Terjesen, B. F., & Summerfelt, S. (2016). Production of market-size North American strain Atlantic salmon Salmo salar in a land-based recirculation aquaculture system using freshwater. Aquacultural engineering, 74, 1-16.

EURE (Santiago) vol.38 no.115 Santiago set. 2012

Harmon, P. R., Glebe, B. D., & Peterson, R. H. (2003). The Effect of Photoperiod on Growth and Maturation of Atlantic Salmon (Salmo Salar) in the Bay of Fundy: Project of the Aquaculture Collaborative Research and Development Program. Fisheries and Oceans Canada.

At the country level, both grape and wine production is concentrated in two regions: the Maule region and the O’Higgins region, with annual growth rates of 3.15% and 7.40% respectively. However, there is a great threat to this growth associated with the exponential development of diseases in vineyards that can lead to lower production and loss of competitiveness in the market. 

At world level, the United Nations Food and Agriculture Organisation estimates that annual losses due to pests are between 20% and 40% of global production, which translates into around 232 billion dollars per year. Along the same lines, the Fundación para la Innovación Agraria de Chile, in its report Detección de virus y fitoplasmas en vid, points out that:

• It has been observed that the diseases produced by these agents produce a reduction in production of around 23%. This has as a direct consequence a very significant drop in profitability, incurring a reduction in the net margin that would lead to an unbearable deficit for the producer. This situation is particularly compromised in the O’Higgins region, since according to data from the Oficina de Estudios y Políticas Agrarias (ODEPA), more than 76% of vineyard holdings are on land of less than 20 hectares, which could mean, in general, a lower financial capacity to face such significant economic losses.
• Over the last few decades, much of the area covered by vines has been replaced by new varieties using the same soils and infrastructure, practices that have led to an increase in populations of nematodes, insects, fungi and bacteria.

To face this threat, the best approach is to be able to carry out an early, rapid and reliable detection of diseases that may affect vineyards, as this will enable early corrective measures to be designed and adopted consisting of the rapid isolation and removal of affected specimens, thus preventing these diseases from spreading throughout the vines and causing significant losses associated with production. 

The limitation in adopting this approach is that there is currently no technology available on the market for diagnosing the health condition of vineyards with high precision, at a low cost and in real time.  In fact, wine producers currently rely on visual inspection and expert assessment to control the development of these diseases, as the alternatives are expensive and not always available, as well as being techniques with low reliability.

Our solution to the problem: disease detection through the processing of multispectral images acquired by a drone.

As we have already indicated, most wine producers carry out the inspection task by visual observation and expert judgement.  This form of inspection consists of carrying out a general observation of the vineyards, trying to visualise, based on empirically acquired subjective criteria, whether the plant has any symptoms that indicate its disease. Doing it in this way involves a great amount of time and resources since direct observation depends on the visual skills of the expert in vineyard diseases who carries it out and his subjective assessment at that moment (a single inspector may take hours to inspect only one hectare). In turn, this method has a relatively high margin of error, since the detection of symptoms or anomalies in plantations depends on the subjectivity of the inspectors, the height and size of the vine, or the climatic condition, among others.

Other non-intrusive techniques available for the detection of diseases are those predictive models that study the evolution of fungi in the grapevine in order to predict infections. These models use meteorological variables such as temperature, rainfall and relative humidity. For example, in the specific case of Mildew (Plasmopara viticola fungus) – a disease well known by winegrowers due to the serious damage it causes to the vineyard – the proposed system is based on the Goidanich model. This model collects meteorological information (average temperature and relative humidity) and estimates when the asexual fruiting of the fungus will occur, which is the ideal time to treat the vine with products to prevent the germination of zoospores. The major limitation of this technique is a very high relative error. 

On the other hand, as regards intrusive techniques for detecting diseases in vineyards, the technological tools developed are based on the combined use of molecular techniques for detecting viruses (ELISA and RT-PCR) and cytoplasms (nested-PCR) in wine and table grapes. This consists of field sampling of the vegetations to be analysed and their subsequent processing and laboratory analysis. It should be noted that the ELISA (Enzyme Linked Immune Sorbent Assay) technique is used as a support (in the case of virus detection) for the PCR (Polymerase Chain Reaction) gene technique, which is suitable for the detection of both viruses and cytoplasms. The limitation of this way of analysing the infection of the vineyards is its very high cost and time consuming.

Based on this reality, together with the wine producers themselves, we identified the need to develop a disruptive solution capable of overcoming the technical and economic limitations of these existing vineyard inspection practices both nationally and internationally.

Our methodology for developing the solution

Through our methodology, starting from the requirements defined as a result of the analysis and research carried out, we were able to define the best approach to solve the problem, specifying technical performance and carrying out a cost-benefit analysis. Thus, we proposed which should be the requirements, which we list below: 

• Measurement of large areas rapidly and efficiently.
• Discretization of the cultivation area into individual elements and obtaining relevant parameters in an automated way.
• Classification of the discretization into “affected” and “not affected”, also indicating type of disease and progress.
• Highly accurate information within 24-48 hours and at a low cost, enabling decisions to be taken at early stages.
• Non-intrusive system, with no interference with the crops. 

Based on this approach, and after an exhaustive analysis of the state of the art, we determined that the best approach was to develop a comprehensive system for monitoring the health condition of vineyards (disease detection) using spectral images captured by unmanned aircraft (drones).

Diagram of our solution for diagnosing the health condition of vineyards

The concept behind this innovative solution is to investigate physiological disturbances by registering changes in leaf reflectance in the near-infrared part of the spectrum, which is not visible to the human eye. Using this approach we can analyse biotic and abiotic stress from differences in the spectral characteristics of the crop canopy. 

In order to identify those spectral parameters that are most characteristic for the detection of diseases in vines, we carried out an analysis of different scientific articles that allowed us to notice that one of the most used indexes is the NDVI (Normalized Difference Vegetation Index). This is a good parameter to evaluate the chlorophyll content of the leaf, which is directly related to the health condition of the plants. Furthermore, from the information collected, it has been possible to implement 8 spectral indexes that can be calculated on each individual tree.

Therefore, returning to the configuration and operation of the solution, the acquisition system (drone) performs a flight over the ground at an altitude of approximately 30 metres using an unmanned aerial vehicle equipped with a five-band multispectral camera.

Tests for validation of the drone equipped with the multispectral camera

After the flight of the drone equipped with the multispectral camera, the data is sent to a cloud server. Using the spectral images, an orthophoto is generated and an atmospheric correction is made. After this, the trees are identified and the spectral indexes of each unit are calculated, with a neural network being used to assign the health condition. 

Therefore, the treatment of the images captured in the drone flight is carried out through a specific algorithm developed to be executed in different steps. 

• The first step consists of processing and correcting the images obtained, since this data must be prepared in such a way as to eliminate the disturbance caused by the spatial variations that occur in the information acquisition process, and by the need to adjust the spatial information to a given reference system. Thus, the ortho-rectification of the multispectral images is achieved and the atmospheric correction allows the calibration of the radiation values to reflectance using the total incoming irradiance.

Example of orthophoto image

• Once this was achieved, the next challenge was to identify and count the different trees by analysing high definition images taken from a drone, to identify each individual tree, obtaining a geolocated map of them and counting the quantity in a given area.

Results of detection and counting algorithms

• As a third step, representative variables were selected and the health condition of the vineyards was classified according to the diseases to be identified. To do this, the indexes and parameters that may represent health condition were taken into account, such as NDVI (Normalized Difference Vegetation Index). 

Image with the 8 spectral indexes on the trees

For this purpose, a classification algorithm was developed, based on multi-layer perceptual neural network that are trained using the maximum likelihood training method, which will select the most representative parameters of each disease, as well as a weighting coefficient, based on successive trainings carried out in the field. 

After obtaining the diagnosis of diseases through the described processing, it is delivered to the client through a digital platform so that the vineyards can be managed. This platform allows the health condition of the analysed plantations to be visualised and positioned on the map, providing geometric characteristics, a map of spectral indexes, identification of the health condition of each vine, filtered on the map by diseases and general statistics and historical data.

In addition, within 48 hours, the client receives a report of the flight made, a document in electronic format with the results of the flight, which includes discretization of the vines, information and geolocalised location of the vines affected by any disease and the progress of the same together with a possible plan of action depending on the disease detected.

Conclusions

Our team has developed an integrated software and hardware system that allows to diagnose the health condition of the vineyard crops in real time, in an accurate way, and at low cost, resulting in a great help to the producers who want to reduce the risk of spreading diseases and the consequent economic losses.

Thanks to the application of Artificial Intelligence, the analysis of spectral images captured by drones are processed by means of complex algorithms that include neural networks, in order to develop a system for diagnosing the health condition of vineyards capable of detecting whether the vine plants analysed suffer from any type of disease, what type of disease affects them and their progress.

For more information on this innovative solution, please download our specific brochure about it. 

In addition, if you are looking to improve the productivity of your vineyard, do not hesitate to consult our other solutions for this industrial sector or contact us to jointly develop a tailor-made technological solution that responds to your specific needs. 

References

Carolina Buzzetti Horta (2018). Una mirada al mercado vitivinícola nacional e internacional.

Fundación para la Innovación Agraria de Chile (2015). Experiencias de innovación para el emprendimiento agrario. Resultados y lecciones en detección de virus y fitoplasmas en vid: proyecto de innovación entre IV región de Coquimbo y VII región del Maule.

Martín, M. A., & Mimbrero, M. R. (2015). Desarrollo e implementación de un sistema para detección temprana de enfermedades en vid en entorno R.

Fundación para la Innovación Agraria Chile (2009). Experiencias de innovación para el emprendimiento agrario. Resultados y lecciones en detección de virus y fitoplasmas en vid: proyecto de innovación entre IV región de Coquimbo y VII región del Maule., 31.

Conveyor belts are a transport system formed by a continuous belt that runs between two pulleys with an intermediate take up pulley. They operate by friction between the belt and the surface of a pulley, which in turn is driven by a motor. The other pulley usually rotates freely, without being driven, and its function is to serve as a return to the belt. The belt is supported by intermediate rollers or idlers, both for guidance and return, between the two pulleys. 

Therefore, conveyor belts are made up of a series of elements such as the head or drive pulley (motor, reducer, bearings, etc.), the tail pulley, the idlers and the belt, among others.

Overview of the belt conveyor.

One of the main challenges faced by conveyors belts is the accelerated deterioration caused by the environment and operation itself, which translates into obstructions and erosion in the mechanical systems due to the deposit of dust particles or other elements, resulting from the mining or port activity. Undoubtedly, this produces a reduction in the useful life of the elements that compose them, leading on many occasions to unexpected downtimes. 

The economic investments to maintain conveyor belts in good condition are enormous. According to studies carried out by international experts, the maintenance cost of conveyor belts can represent up to 30-50% of the operating cost of a mining operation. 

To give an example showing the order of magnitude of the maintenance investments necessary also in the port terminals, at the Europees Massagoed Overslagbedrijf BV terminal in Rotterdam, maintenance cost of conveyor belts reached 5 million dollars only in 2013, for a total turnover of 175 million dollars. 

Entering into a more comprehensive analysis, about 52% of this investment is used for the maintenance of the rotating elements (drive pulley or tail pulley, among others), 18% for the intermediate idlers, 4% for the belt and 26% for the remaining components. It should be noted that of the 52% maintenance cost of the rotary elements, 46% is due to bearing failures. There was no technology on the market before our solution that could reliably detect failures in this element.

In addition to maintenance, the operator must consider the cost associated with downtime of a conveyor belt due to unscheduled stops caused by failures in different elements. According to personnel at the Queensland Gold Mine in Australia, one hour of conveyor belt downtime can result in economic losses of 40,000 – 50,000 dollars. 

If we analyse the causes of unplanned conveyor belt downtime, these are mainly due to pulley failures (54% of the total downtime), with 28% corresponding to intermediate idlers, 9% to the belt and 9% to other elements.  

Given the lack of solutions on the market capable of providing an optimum solution to this situation, we identified the need to develop a digital tool capable of characterising the defects of all the elements of the belt in early stages, making it possible to move from the current corrective maintenance approach (acting once the failure has occurred) to a predictive one (in which the damage can be anticipated and, consequently, action can be taken before the critical failure).

Our solution to the problem: real-time detection and prediction of failures using temperature and inertial response

Currently in Chile, conveyor belt maintenance, both in the port and mining areas, is traditionally carried out based on the results of visual inspections and corrective maintenance. However, new practices are emerging in the market that combine visual inspections with different non-destructive inspection technologies, based on thermography, ultrasound or vibration analysis, with the aim of reducing the limitations and subjectivity associated with traditional practices. 

However, these techniques have important limitations. For example, solutions based on thermography, which study the temperature levels of machines in order to detect electromechanical problems, have as their main limitation their inability to diagnose the problems, since they only detect them once they are in highly advanced states.

On the other hand, there are solutions based on ultrasound, which are based on the analysis of high-frequency sound waves produced by the machines when they experience a problem. However, its use is limited to bearings and is not capable of either detecting the reason for the failure or providing a diagnosis.

Finally, there are solutions based on vibration analysis, which rely on the study of the vibratory signals emitted by rotating machinery, allowing certain signal processing methods to be applied to extract the information contained in these vibration registers. Its main limitation is the masking of the signals, presenting low accuracy in detecting faults (around 40-50%).

Considering the limitations of the existing systems and the technologies presented in the state of the art, we focused on developing our own solution which, through the development of filters and advanced mathematical processing of data recorded in situ, would be capable of achieving precise diagnoses in early stages of all the key elements that compose a conveyor belt.

Our methodology for developing the solution

Through our methodology, taking as a starting point the requirements defined as a result of the analysis and research carried out, we decided to develop a global monitoring system for the different elements of the belt that would make it possible to carry out a diagnosis in a simple, automatic, wireless and real-time manner, while generating a predictive maintenance plan.

Based on this approach, and after an exhaustive analysis of the state of the art, we determined that the best approach was to combine two different technologies: 

• Vibration analysis of the drive pulley, tail pulley and take up pulley; and
• Temperature analysis of the intermediate idlers.

Starting from these premises, and after designing and assembling sensor nodes with accelerometers of the necessary sensitivity and robustness, we developed algorithms to characterize the vibratory signal of certain rotating elements (drive pulley, tail pulley and take up pulley), including a pre-processing (filtering) that would allow removing the noise from the registered signal. 

Thus, in order to unmask the signals that present noise from various sources and could induce mistakes, a self-adaptive filter was developed to eliminate the noise that hinders the analysis of the vibratory signals. This filtering also incorporated an algorithm capable of updating the filtering parameters to minimise error and get as close as possible to the real signal of the elements whose signal is masked (e.g. bearings). 

Finally, after obtaining the processed signal, a pattern identification algorithm was developed, by means of which, with a comparison of vibratory patterns, our solution is capable of determining the existence of a defect, recognising the type of fault that exists and the element in which it occurs.

Installation configuration of the accelerometer nodes in the drive pulley

At the same time, a mathematical algorithm was generated so that the system would be able to obtain, through the registered temperature, if there was an abnormal operation of any intermediate idler.  

The reasoning behind this approach is that when an intermediate idler is in a bad condition due to the wear of the bearings that make it rotate properly, it vibrates. These vibrations produce a temperature increase that exceeds the normal operating threshold. Thus, normal operating temperatures range from 20°C to 50°C, depending on the surrounding temperature. If the temperature of a idler increases at higher temperatures, which range from 80°C to 120°C, then that is a clear sign of a potential malfunction. By incorporating these temperature sensors into the idlers, it was possible to monitor this overheating so that it was possible to detect intermediate idlers that were in poor condition. 

Thanks to the numerous tests carried out during the development of the solution, it was possible to obtain and calibrate different temperature threshold values depending on the position of the idler, its speed of rotation and the external temperature, making it possible to diagnose the defect and its cause.

Installation configuration of temperature sensor nodes in intermediate idlers.

As a final stage in our methodology, the solution was validated on several conveyor belts in Puerto Panul and Puerto Mejillones, being able to demonstrate the high reliability of our solution and its advantages over the competition in both cases.

Image of one of the nodes installed in the drive pulley (left) and installation of the router node by one of our engineers to ensure secure data transmission

Conclusions

Our team has developed an integrated system composed of innovative software and hardware units that makes it possible to detect defects in any of the key elements of the conveyor belts, so that a predictive maintenance strategy can be implemented that minimises the necessary investment and avoids unexpected downtime in all cases.

Thanks to the application of Artificial Intelligence technologies, the characterisation and analysis of vibration patterns and temperature patterns of the different elements makes it possible to diagnose hidden faults at early stages, something that was impossible until now.  

For more information on this innovative solution, please download our specific brochure. 

In addition, if you are interested in improving the productivity of your industrial sector, do not hesitate to check other solutions developed by our team or contact us to jointly develop a tailor-made technological solution that responds to your specific requirements. 

References

Ašonja, A., & Adamović, Ž. (2010, September). The Economic justification of the Automatic lubrication Using. In 14th International Research/Expert Conference” Trends in the Development of Machinery and Associated Technology” TMT 2010, Mediterranean Cruise (pp. 11-18). 

Thieme, K. R. (2014). Economic Justification of Automated Idler Roll Maintenance Applications in Large-Scale Belt Conveyor Systems; Economische rechtvaardiging van geautomatiseerde applicaties voor rollenonderhoud in grootschalige systemen van bandtransporteurs.

Zimroz, R., & Król, R. (2009). Failure analysis of belt conveyor systems for condition monitoring purposes. Mining Science, 128(36), 255.