After the construction of a drinking water network, the most important thing is to monitor the maintenance condition of the pipes. The amount of data collected for this purpose is growing enormously, especially when it comes to video inspections. But how do you find useful information as a drinking water utility in such a growing mountain of data? In other sectors, image processing techniques are already widely used for this purpose. Could this also be a solution for the drinking water sector?
Network operators usually rely on ‘manual’ estimates from statistical analyses and asset data to predict the condition of the pipes. In recent years, however, mobile inspection platforms have also been developed. Such devices use cameras and other sensors to assess the network from the inside. Further developments in the field are expected to lead to the collection of ever larger amounts of data. Video inspections, for example, will provide the drinking water industry with millions of images of the inside of pipes.
For water utilities, registering the components of the pipe system – such as pipe joints – is one valuable step in mapping out the condition of the pipe network in more detail. However, continuing the example, there are millions of connections in the pipeline network, making it impossible for humans to search for them among inspection data. In this article, we explore and demonstrate the potential of automatic processing of video inspections by applying it to detect the location of connections. The work is part of KWR Water Research Institute’s project ‘The pipeline network in view’.
Approach
The images in this study were made with two different devices that are already in use at Dutch water utilities. One is an inspection robot (IBAK PANORAMO). The other is a camera mounted on a ‘pig’, moving through the pipe (Quasset). Because the devices are different, they produce different images. Image processing algorithms will have to be able to cope with this. An example of a shot from the IBAK robot is shown in Figure 1A.
The conventional image processing process starts with a pre-processing step that improves the colour contrast. Segmentation is then applied, which allows different elements in the image to be distinguished on the basis of characteristics such as brightness and colour. Then more abstract features, such as shape, centre or diameter, are extracted from these segments. These characteristics are fundamental data for computers to recognise or detect objects. Finally, the useful information extracted from the images is summarised and translated so that system operators can easily understand the state of the system and make decisions based on this information.
These steps are explained below for the inspection of connections.
Algorithm
The first step in the image processing algorithm is to convert the data into grey values. Each pixel is assigned a grey value for the amount of reflected light (Figure 1B). The move to greyscale images makes further data processing easier.
Based on the differences in light intensity, a Canny edge detector algorithm can then identify the edges between objects (Figure 1C). The edges are then artificially enhanced (exaggeratedly dilated) to show a better representation of the objects found. (Figure 1D).
In Figure 1D, several lines can be distinguished, such as the circle marking the connection, as well as the diagonal lines representing the track left by the inspection robot. To enable a computer to detect only the connection, an algorithm is then applied to the dilated edges that is ideally suited for detecting circular patterns: the “Circular Hough Transform” (CHT). Figure 2A shows which circles the algorithm was able to identify in the previous steps. To remove the irrelevant circles, as a final step in the algorithm, a filter is added to exclude those whose centres are not at the centre of the image. Thus, only the circle corresponding to the connection remains, as shown in Figure 2B.
The above algorithm can be applied to any image in a video of the pipe. In some of these images, there is too much noise to be able to detect circles. However, one connection can be seen in several consecutive images; each connection is ultimately detected on the basis of the relationship between neighbouring images.
Mapping the components detected
The algorithm described can indicate in a video which images show a connection and which do not. In order to record the connections in the pipe information system, it is also necessary to know the corresponding location of the robot in the pipe network for each individual image. In order to automatically determine the exact location of pipeline components, in addition to an algorithm for recognising connections and valves, an adequate camera positioning system is required. For other possible inspection data, it is also essential to link the measurement to the relevant location and pipe. Only then can water utilities sharpen their decisions about specific pipelines on the basis of the inspections.
Limitations of ‘classical’ image processing
In the case study above, we can see that these conventional image processing techniques can already support us enormously in processing large amounts of inspection data. However, there are also limitations.
The successful application of image processing in this case is due to the fact that the algorithm is focused on the precise characteristics of the target: a circular joint in the centre of the image. However, the algorithm is not flexible enough to allow it to be used for other things that are worth detecting, such as the locations of drillings, incipient cracks or deterioration. The algorithm for connections uses the fact that all connections generally look the same. The shapes of cracks and other signs of degradation are much more diverse, which also makes it more difficult to design a targeted algorithm.
An even more important limitation is that an algorithm can only be focused on a detectable aspect if we know what exactly we are looking for. This means that we cannot detect unknown abnormalities in a targeted way. Since video inspections of drinking water pipes are not yet widely used, we do not yet have a good picture of all the issues that may be worth detecting. Thus, a lot of “manual” experience is still needed before image processing algorithms can take over tasks.
Potential of ‘modern’ artificial intelligence
More modern artificial intelligence (AI) techniques are better suited for detecting more diverse or unknown objects. With AI, a self-learning system can be taught to recognise objects on the basis of examples (machine learning) and then also recognise new variants of these objects. In other sectors, it has already been demonstrated that such algorithms can eventually become even better than humans at detecting objects. It is important to emphasise, however, that thousands or even millions of examples must be provided to such algorithms before they can function on their own. Although artificial intelligence can be a very powerful tool for detecting unknown or diverse objects, it is not a substitute for human experience. The drinking water sector will first have to explain to the computers themselves what they have to watch out for.
Next steps
This article shows by way of an example that image processing techniques can be used to automatically recognise objects in video images of water pipes. The Circular Hough Transform method was used to detect connections – this method has proven effective in identifying circular objects. The images of the detected connections must also be linked to the measurement location. Such forms of automated data processing are crucial for the widespread use of video inspection of pipes.
The first important step towards using computers to process inspection data in the future is to gain experience and find out exactly what we want to get out of our inspections. On the basis of these insights, classical or more modern image processing techniques can then be set up and adjusted. In follow-up research, therefore, it is important to have more inspection images searched through by real people.
Mollie Mary Torello, Xin Tian, Peter van Thienen, Karel van Laarhoven (KWR Water Research Institute)
Sources
KWR Water Research Institute, website consulted on 12-05-2022. www.kwrwater.nl/projecten/het-leidingnet-in-beeld/
