 Research article
 Open Access
 Published:
Unsupervised learning approach to automation of hammering test using topological information
ROBOMECH Journal volume 4, Article number: 13 (2017)
Abstract
In this paper we present an online unsupervised method based on clustering to find defects in concrete structures using hammering. First, the initial dataset of sound samples is roughly clustered using the kmeans algorithm with the kmeans++ seeding procedure in order to find the cluster best representative of the structure. Then the regular model for the hammering sound, the centroid of this cluster, which is assumed to be the nondefective sound model, is established and finally used as a reference to conduct diagnosis on the whole dataset. During the model generation phase, topological information on the spatial distribution of samples is used to attribute varying importance to each sample and therefore take into account meticulous diagnosis of certain areas. The algorithm is fast and reliable enough to allow efficient diagnosis by running it each time a new sample is acquired. Tests on two commonly found types of defects, namely delamination and void type defects, were conducted on experimental test blocks and yielded satisfying results. This method also performed well in field environments.
Background
The designation concrete covers a large spectrum of composite materials composed of aggregates bonded by a fluid cement, hardened over time. This material is extremely common in modern societies, especially in social infrastructures such as tunnels. As any other material, concrete can be greatly affected by aging and environmental conditions. In some cases, these factors may lead concrete structures to structural failure [1,2,3]. In order to guarantee their safe use, careful maintenance is needed. Among all the operations taken to maintain these structures, the diagnosis for defects is critical since it is a decisionmaking step.
Among all the available nondestructive testing methods [4], a popular one for concrete structures is called hammering test (Fig. 1). In this study, we focused only on one variation of hammering called “tapping”?. This method, consisting of an operator perpendicularly hitting one point on the surface of the structure with a hammer and assessing the presence of defects from the perceived sound, has the advantages of being nondestructive and not needing heavy equipment. However, it requires a skilled operator to be able to correctly analyse the sound and given the huge population of structures in need of examination currently in service [5], testing them all with this traditional method reveals to be problematic. Therefore, the automation of the hammering test is demanded.
Various attempts to adapt the hammering test in an automatic form have been made in order to obtain a faster, more reliable and objective method to find defects in concrete structures. References [6, 7] were focused in finding sound features enabling differentiation between defective and nondefective spots as well as on the exploration of new methods to replace or aid the human operator holding the hammer in order to get more regular and reliable sound samples. References [8,9,10,11] were more focused on the data analysis part of the problem and used supervised learning to correctly distinguish sounds from nondefective areas and sounds from defective areas. These approaches have given promising results, however their main drawback is the necessity to train the algorithm first using a training set. Depending on various factors, especially during the hardening phase, concrete can greatly differ from one structure to another, even if they were made from the same batch, thus choosing the adequate training set can be difficult. Our proposed method takes a new approach to this task using unsupervised learning, based on clustering, and therefore bypasses the need of training sets.
In our previous method [12], the major cluster from which the centroid to be used as model was determined by finding the cluster with the biggest number of samples in it. This method was acceptable when diagnosis was conducted following a regular grid. However, in practice, defective areas of the structure are more carefully examined: this could result in having more defective samples in the dataset and nondefective ones and thus, the method wrongly choosing the wrong centroid as regular model. In this paper, we introduce a weight system to balance the respective influence of samples in accordance with their spatial distribution on the tested structure surface. This enables proper balancing of each sample’s relevance given their mutual spatial proximity.
Concrete structure inspection is generally divided into two stages [13]. The first one, called primary inspection, is a rough one, conducted on the whole structure. If any defect is found during this process, the secondary inspection is conducted only at these spots to accurately identify the defect. Given the nature of our proposed method, adaptable and based on statistical irregularity detection, it can be considered particularly suited for the primary inspection of concrete structures.
In this paper, we propose a method to allow training setfree, realtime, adaptive hammering testing of concrete structures. This allows hammering testing to be conducted on a unknown structure, obtaining a primary inspection diagnosis and narrowing down spots for secondary inspection.
Methods
Concept
The main assumption is that most of the tested structure is nondefective. That means that defects, such as cracks and voids, do not occupy the majority of the tested surface. This assumption is acceptable since concrete structures subject to severe deterioration are blatant and therefore do not require to be tested, a simple inspection by naked eye is enough. Considering this, it becomes possible to characterize the nondefective sound as the regular sound found on the tested surface.
This approach has been motivated by two main reasons. First, interview conducted with actual professionals in charge of conducting hammering tests revealed that they were, in fact, more focused in hitting multiple spots on the structure at high speed and looking for sounds that stands out rather than relying in past experiences and knowledge. In this aspect, our proposed method is closer to what human operators do. Secondly, as stated earlier, it has been observed that concrete is extremely sensible to physical conditions such as temperature, humidity, etc., that especially during the hardening phase of the fabrication process. The result is that even among concrete blocks that were made from the same source, nondefective spots do not return similar hammering sounds at all. This was observed at various occasion with concrete test blocks we used in our experiments: even tough they were made in a single batch, nondefective spots in two different blocks returned different sounds.
More precisely, given an initial dataset of hammering samples, we can distinguish 4 steps in our proposed method as illustrated in Fig. 2:

1.
Regroup hammering samples that are similar.

2.
Find the major group of the tested structure.

3.
Extract the centroid of this group and establish it as the regular, nondefective model.

4.
Use the generated regular model as a reference to conduct diagnosis on the samples.
Regrouping hammering samples
Description of a hammering sample
In this paper, two elements are used to define a hammering sample:

1.
The recorded sound.

2.
The location where the sample was recorded.
As it is usually done when handling audio data, Fourier spectrum is used as feature vector for a hammering sound.
Given a sound sample \(X_{i}\) defined by (\(x_{0},\ldots,x_{N1}\)), N being the sampling rate multiplied by the recording duration, collected on a particular location \(\mathbf {L}_{i}\) on the structure, its Fourier spectrum (\(a_{0},\ldots,a_{N1}\)) is obtained using Fast Fourier Transform (FFT).
Metric
In order to compare sounds, i.e to give a value of how much two hammering sounds are similar, a meaningful distance measure between sound samples in the Fourier spectrum space has to be defined.
Given two Fourier spectrum A and B, respectively defined by (\(a_{0},\ldots,a_{N1}\)) and (\(b_{0},\ldots,b_{N1}\)), the sample Pearson correlation coefficient is defined as in Eq. (1).
The sample Pearson correlation coefficient has the advantage of providing a zero mean and unit standard deviation normalizations. Features of the Fourier spectrum of each sound sample used for comparison are only related to the general shape and amplitude variations, directly influenced by the applied hammering force, are not taken into account. Therefore, it can be considered robust towards changes of the force applied by the human operator of the hammer that induces sounds of different amplitude being recorded, i.e. this mitigates the need to take into account the input to the system for our method and a simple hammer can still be used for testing.
The sample Pearson correlation coefficient ranges in [−1,1]. Negative values signifies a negative correlation, and positive values corresponds to correlation. Values close to zero implies there is no correlation between the two samples. We can define a distance measure based on this coefficient, a correlation distance, as in Eq. (2).
The defined distance is ranging in [0,1], returning small values the more the compared sounds are alike and zero if the sounds are identical. Cases of negative correlation are located in the [0.5,1] range since negative correlation is in our case not a similarity.
Rough clustering using kmeans++
In our case, we do not need quality clusters since clustering is not our goal, i.e. clustering is not conducted as a step for further cluster analysis. The final aim is to obtain the regular model. Moreover, to work toward a system with the capability to conduct diagnosis in realtime, a fast algorithm would be useful. Considering the usual dataset of hammering samples, usually around a few hundred in our application, we found the kmeans algorithm, usually used in data consolidation or preclustering, being adequate: it is simple and computationally fast enough.
Even if defective samples can be spread out in feature vector space (defects are unpredictable and can be of several variations in a single structure), more compactness can be expected for nondefective samples: kmeans should not fail to put at least one centroid in the nondefective sample group.
In order to obtain more consistent runtime for the kmeans algorithm, the kmeans++ seeding procedure [14] is used. In our proposed method, only two clusters are needed, therefore the procedure is applied for only two seeds. The first seed is chosen randomly following an uniform distribution. For the second seed, a probability distribution to reflect the similarity to the first seed is devised: each sample \(X_{i}\) has a probability \(P(X_{i})\), as defined in Eq. (3), based on the previously defined metric to the first seed \(S_{1}\), \(d(X_{i},S_{1})\), to be chosen. Unlike the regular seeding process where the seeds are simply chosen randomly, this procedure allows the seeds to be spread trough the dataset and therefore close to the final centroids location. Other than the acceleration of the algorithm for large datasets, the advantage to use this seeding procedure is that it stabilizes the run time by not being entirely random such as the regular kmeans seeding method.
With both the metric and the seeding procedure defined, kmeans is used to cluster the dataset of hammering sound samples, transformed into Fourier spectrums, into two clusters.
Finding the major cluster
In our approach to this task, the cluster best describing the tested structure has to be found.
Defining the regular model based on the number of sample contained in each cluster would be enough in the case of samples being collected following a grid: each sample would then have the same weight in the final comparison. However, when collecting samples freely on the structure, defective areas tend to be tested meticulously and therefore, the number of samples from defective spots tend to surpass the number of samples from nondefective spots. If this happens, such simple implementation would wrongly recognize the defective sound as being the regular sound and thus the nondefective sound model.
To overcome this problem, a weight symbolizing the influence of one sample is defined. More concretely, for each sample can be considered an area around it where it is the representative of. In this paper, we simply choose to define this area as a disc centered on the location of the corresponding sample \(\mathbf {L}_{i}\) (Fig. 3). This location is simply collected by painting the hammer head in red and tracking it using image processing: when a hammering sound is detected by the trigger and recording begins, the location of the hammer head at that moment is saved along with the sound data.
Given a sample \(X_{i}\) contained in dataset D among \(N_{sample}\) samples, with its location \(\mathbf {L}_{i}\), the radius of the sample’s area is defined as half of the Euclidian distance to its nearest neighbor:
The surface of this disc is then used as a weight for this sample:
With this method, the importance of samples is dependent of their topological distribution and therefore, it is possible to conduct a diagnosis that takes into account what has been already obtained in the previous runs of the method.
To determine the cluster containing most of the nondefective samples, the weight devised in Eq. (5) is expanded to clusters: for a cluster \(Cl_{i}\), its weight \(W_{Cl_{i}}\) is defined as the sum of weights of all samples contained within it. This means to represent how much spatial importance this cluster has on the structure.
Then, the major cluster is defined as the one with the biggest weight:
Generation of the regular model and diagnosis
With the regular cluster \(Cl_{major}\), containing \(N_{major}\) samples, found previously, its centroid \(C_{major}\) is used as our regular model:
Each sound sample of the dataset is finally compared to the model using once again the correlationbased distance defined in Eq. (2), i.e. the generated regular model is used as reference to scale and evaluate the samples. Since the model represents the most regular sound shape in the dataset, irregularities, i.e. distant sound samples can be recognized as characteristic of defects on the structure, distance to the regular model can be expressed as a similarity value (Fig. 4). From there, a simple threshold can be enough to determine if a sample is defective or not. Given a threshold value Th, a boolean value defective for a sample \(X_{i}\) could be defined as:
The pseudoalgoritm presented in Algorithm (1) briefly sumsup the mechanics of our proposed method. The clustering step being fast enough, online implementation was possible by simply running it each time a new sample is added to the dataset.
Results and discussion
Experiments using a traditional hammer
The used setup is illustrated in Fig. 5 and experiments were conducted on concrete test blocks containing various manmade defects to simulate natural ones. For each block, defective spots are marked in red on the corresponding schematic.
Test blocks were hit at 210 locations once following a 14 by 14 square grid that covers the whole block. The used hammer was a KTC UDHT2 (head diameter 16 mm, length 380 mm, weight 160 g), commonly used in hammering test by professionals and sound was recorded at 44.1 kHz using a Behringer ECM8000 microphone coupled with a RolandUA25EX sound board and a laptop for data analysis. Fourier spectrums were computed by FFT with a window of 1024, thus in vectors of length 512 due to symmetry.
A simple trigger was implemented to conduct clipping to get each hammering sound as a single sample and the hammer head was painted in red in order to be tracked so that the Cartesian coordinates of each sample could be collected.
Delaminationtype defects
Delamination is a phenomenon mostly observed in reinforced concrete structures. These structures are often subject to reinforcement corrosion: the reinforcement metal is oxidized and its volume increases. This results in internal stresses in the concrete structure and the apparition of cracks diagonal to the surface of the structure.
Tests were conducted with \(500 \times 500 \times 150\) mm concrete blocks presenting cracks at an angle of respectively 15°, 30° and 45°. The schematics of these test blocks are presented in Fig. 6. Using these schematics as ground truth, receiver operating characteristic (ROC) curve was calculated for each of these blocks (Fig. 7) by varying the threshold value Th used in Eq. 9. It can be noticed that the area under the ROC curve decreases from 0.94 to 0.80 as the crack angle increases from 15° to 45°. This could be attributed to the defect depth: the bigger the crack angle, the deeper the crack runs in the concrete, resulting the sound to be more and more muffled as the crack angle increases, rendering analysis more difficult (Table 1).
In [15], Computer Vision is used to detect cracks on the surface of concrete. Depending on the used preprocessing techniques, the area under the curve varied from 0.87 to 0.99. Our proposed method could therefore be considered acceptable.
In Fig. 8 the correlation distances from the model for nondefective and defective hammering samples of the 15°. delamination test block is shown (in total 210 samples). Except a dozen defective samples that were incorrectly given small distance to the model, it can be observed that our proposed approach successfully separates the two types of samples.
Always for the 15° delamination defect test block, in Fig. 9 the Fourier spectrums of the computed model for nondefective sample, a nondefective sample and a defective sample are shown. It can be seen that the produced model is a Fourier spectrum that more closely resembles the Fourier spectrum of a nondefective sample than of a defective sample.
Time taken for the algorithm to return a result at each new sample was measured. After stabilization, when around 200 samples were collected, the average processing time for a new sample was 454 ms. This allows the hammer to hit the structure about twice in one second and therefore enables our proposed method to keep up processing when hammering is done manually. Since hammering is usually conducted locally, around a suspected area on the structure’s surface, the dataset of samples is not expected to grow up to very large sizes and therefore it can be considered that, in the scope of our application, it is satisfying.
Voidtype defects
For various reasons such as a foreign body or the infiltration of water/air, defects we could call voidtype can appear on concrete structures in the field. Basically, during the hardening phase, the concrete was not able to completely fill the required volume. This type of defects is considered extremely dangerous since for one part they tend to make large portions of concrete beneath them to be instable and fall off, and for the other part because unlike delaminations, they are usually not visible at the surface of the structure. The hammering test is in this case a trusted tool for inspection.
A test block containing cuboidshaped polystyrene bodies were used to simulate this type of defect (molding a block of concrete with a controlled volume of air inside was impossible). Its schematics is presented in Fig. 10.
As with the delamination’s case, the ROC curve was established on this block (Fig. 11) for defects of depth 30, 50 and 75 mm. Due to abnormality during production, the defect at 15 mm depth was not available for testing. Again, the value of the area under the curve were calculated (Fig. 11) and are shown in Table 2.
Experiments on the field
Our proposed method was tested under field conditions at an experimental tunnel made in order to naturally present defects. It is worth noticing that these defects are different from the ones on the previously presented test blocks since, though induced, they occurred naturally. Therefore, no ground truth was available in order to quantitatively evaluate our proposed method. However, in order to effectuate a qualitative evaluation, a professional operator conducted hammering and our proposed method was tested on the found defects. The schematic specifying the dimensions of this tunnel is presented in Fig. 12 and the picture in Fig. 13 shows both our setup for data collection and the general area were defects were concentrated.
The tested spot on the tunnel was a void type defect identified on the side of the tunnel, at a height of approximately 4 m. The result obtained then is shown in Fig. 14, the threshold value Th was manually adjusted in order to get the best discrimination of defective samples.
Conclusions
Our proposed method was able to successfully identify both delamination and void type defects without the need of any training set, in a realtime fashion and by allowing adaptive hammering testing. The method was also able to show a similar performance on the field, on an unknown and untested structure, and to this regard, it could be judged adequate for the purpose of primary inspection, successfully narrowing down areas for secondary inspection. For future work, we would like to improve this method to ensure increased robustness, especially on the field where sources of noise such as wind are abundantly present. Another point worth further investigating would be on the hammering force; although its influence can to be considered to have been mitigated by normalizations in our devised metric, there is no guarantee that it does not influence the shape of hammering sound spectrums. Therefore, it would be interesting to either measure the hammering force and incorporate this aspect in the diagnosis. Also, with the recent development of automatic hammering modules, that would enable consistent hammering with the same force for the whole structure, the performance of this method in combination with these robots should also be investigated.
References
 1.
Mori Y, Ellingwood BR (1993) Reliabilitybased servicelife assessment of aging concrete structures. J Struct Eng 119:1600–1621
 2.
Cabrera J (1996) Deterioration of concrete due to reinforcement steel corrosion. Cem Concr Compos 18(1):47–59
 3.
Basheer P, Chidiact S, Long A (1996) Predictive models for deterioration of concrete structures. Constr Build Mater 10(1):27–37
 4.
Hoła J, Bień J, Sadowski Ł, Schabowicz K (2015) Nondestructive and semidestructive diagnostics of concrete structures in assessment of their durability. Bull Pol Acad Sci Tech Sci 63(1):87–96
 5.
Takada R, Oishi N (2013) Priority issues of infrastructure inspection and maintenance robot: a part of cocn 2012 project disaster response robot and its operational system. In: 2013 IEEE region 10 humanitarian technology conference (R10HTC). pp 166–171
 6.
Sonoda Y, Fukui Y (2010) A basic study on hammering tests of deteriorated concrete structures. In: Proceedings of our world in concrete and structures. p 100035063
 7.
La HM, Gucunski N, Kee SH, Yi J, Senlet T, Nguyen L (2014) Autonomous robotic system for bridge deck data collection and analysis. In: 2014 IEEE/RSJ international conference on intelligent robots and systems (IROS 2014). pp 1950–1955
 8.
Zhang G, Harichandran RS, Ramuhalli P (2012) An automatic impactbased delamination detection system for concrete bridge decks. NDT E Int 45(1):120–127
 9.
Fujii H, Yamashita A, Asama H (2014) Automated diagnosis of material condition in hammering test using a boosting algorithm. In: 2014 IEEE workshop on advanced robotics and its social impacts (ARSO). pp 101–107
 10.
Zhang G, Harichandran RS, Ramuhalli P (2011) Application of noise cancelling and damage detection algorithms in nde of concrete bridge decks using impact signals. J Nondestruct Eval 30(4):259–272
 11.
Jiaxing Y, Iwata M, Takumi K, Murakawa M, Tetsuya H, Kubota Y, Yui T, Mori K (2014) Statistical impactecho analysis based on grassmann manifold learning: Its preliminary results for concrete condition assessment. In: EWSHM7th European workshop on structural health monitoring. pp 1349–1356
 12.
Kasahara JYL, Fujii H, Yamashita A., Asama H (2015) Unsupervised learning approach to detection of voidtype defects in concrete structure using hammering and clustering. In: 6th International Conference on Advanced Mechatronics (ICAM2015) (2015)
 13.
Asakura T, Kojima Y (2003) Tunnel maintenance in Japan. Tunn Undergr Space Technol 18(2):161–169
 14.
Arthur D, Vassilvitskii S (2007) kmeans++: the advantages of careful seeding. In: Proceedings of the eighteenth annual ACMSIAM symposium on discrete algorithms, society for industrial and applied mathematics. pp 1027–1035
 15.
Fujita Y, Yoshihiko H (2011) A robust automatic crack detection method from noisy concrete surfaces. Mach Vision Appl 22(2):245–254
Authors' contributions
JYLK developed the method and drafted the manuscript. HF was involved in discussion of ideas and results and manuscript drafting. AY and HA supervised the project and provided guidance. All members are involved in checking and approval of the paper. All authors read and approved the final manuscript.
Acknowledgements
This work was supported in part by the Crossministerial Strategic Innovation Promotion Program (SIP) of the New Energy and Industrial Technology Development Organization (NEDO), GrantinAid for JSPS Fellows 269039, and Institute of Technology, Tokyu Construction Co., Ltd.
Competing interests
The authors declare that they have no competing interests.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Author information
Additional information
Jun Younes Louhi Kasahara, Hiromitsu Fujii, Atsushi Yamashita and Hajime Asama equally contributed to this work.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Received
Accepted
Published
DOI
Keywords
 Clustering
 Hammering
 Concrete
 Online
 kmeans