1.1 Copy-move detection

Various state-of-the-art approaches are available to identify copy-move alterations. However, for most of the algorithms studied, the detection of forgeries consists of four main stages: preprocessing feature extraction, pairing, and visualization [⁴]. One of the most used methods for copy-move forgery identification is the Scale-Invariant Feature Transform (SIFT), as this method works on the image’s key points extraction. In [⁵], authors presented a hybrid method for detecting copying on an image using SIFT and the Principal Component Analysis (PCA) kernel to extract the main points by blocks. A variation of the previous method was presented in [⁶], in which they used the KAZE point detector together with SIFT to extract more crucial points. A method to identify copy-move forgeries with the Discrete Wavelet Transform (DWT) was proposed in [⁷]. Initially, they applied the DWT on the image to obtain both the detail and approximation coefficients; then, they extracted the critical points with SIFT on the detail coefficients. Subsequently, they compared the feature vectors to identify which regions were falsified. Another recognized feature extractor is the Speeded Up Robust Features (SURF), which detects interest points and descriptors [⁸-⁹]. A technique to detect copy-move forgeries based on SURF and KD-Tree for the comparison of multidimensional data was presented in [¹⁰]. Recently, many researchers applied deep learning methods in areas such as mechanics [¹¹], medicine [¹²], and a solution for copy-move detection [¹³]. [¹⁴]proposed a deep learning approach with a transfer learning model that uses VGG-16 custom design convolutional neural network (CNN). A deep learning technique based on the CNN model with multi-scale-input and multi-stages of the convolutional layer was proposed [¹⁵]. This method used three phases: encoder block, decoder block to extract feature maps and classification. In the encoder block, the images are downsampled in multiple levels to extract features maps. In the decoder block, features maps are combined and upsampled until the output feature's dimension matches the input image's dimension. Finally, a sigmoid activation function was adopted as a classifier in the last phase. The authors in [¹⁶] suggested a dual branch CNN model with multi-scale input by choosing different kernel sizes. First, the images were resized and standardized on-the-fly; then, the images were passed to the CNN. The CNN architecture had two branches with standard input but different kernel sizes to extract distinct features maps. From the experimental result, the method was lightweight and performed high-grade prediction accuracy for the MIC-F2000 dataset.

1.2 Resampling detection

Several authors have focused their work on the identification of unnatural periodicities in digital images to detect resampling alteration. A method to detect the periodicity in images introduced by resampling and the compression of the JPEG format was implemented. To achieve this, they calculated the probability map of the image using the EM (Expectation-Maximization) algorithm [¹⁷]. Another method for resampling detection based on the EM algorithm and the probability mapping (m-map) of pixels in the frequency domain was used in [¹⁸]. This technique worked in the absence of any watermark or digital signature. In [¹⁹], a method capable of detecting traces of geometric transformation was proposed, using the periodic properties present in interpolated signals to detect whether the image has been modified or not.

1.3 Splicing detection

For this type of falsification, the paste operation alters the image’s general statistics by incorporating textures and borders that contrast with the original image [²⁰,²¹]. In recent years, some work carried out for splicing detection has focused on the extraction of the characteristics of an altered image and its original. The most relevant characteristics are used to train and validate a classifier, then the trained model is used to detect forgery on the image [²²,²³]. In [²⁴,²⁵], the authors presented a Local Binary Pattern (LBP) method due to its ability to represent textures with a low computational cost. LBP and the approximation coefficients of the DWT to extract the frequency characteristics were proposed in [²⁶,²⁷], incorporating the histograms of the DWT detail coefficients to the vector support machine. Several authors have proposed LBP methods, such as LBP-DCT [²⁸] and LBP-Enhanced [²⁹]. In [²⁰], an excellent method to highlight statistical changes from Markov's transition probability characteristics was introduced. Other authors have proposed combinations to improve results, such as Markov-DCT-DWT[³⁰], Markov-DCT [³¹], Markov-QDCT[³²], and Markov-Octonio DCT [³³]. As copy-move detection, a deep learning approach is used for splicing detection. In [³⁴], a two-branch CNN learns hierarchical representations from the input RGB color or grayscale test images and feeds a support vector machine as a classifier.

In this study, a hybrid algorithm for the detection of copy-move, resampling, and splicing forgery is proposed. The main contributions of this work are as follows:

2.1 Markov preprocessing

First, the image was converted to grayscale. Then, the horizontal and vertical derivatives of the image in space were calculated, after which the Markov process was carried out to obtain the characteristics in space. Subsequently, the image was divided into 8x8 non-overlapping blocks, and the DCT was applied to each block. After that, the horizontal and vertical derivatives were calculated for the resulting image. Next, the Markov process was performed to obtain the frequency characteristics. Finally, a vector of spatial and frequency characteristics was created to train a polynomial SVM.

DCT block

Since the altered image had changes in its local frequency distribution, it was possible to detect these changes with the DCT coefficients. First, the grayscale image was divided into 8×8 blocks without overlapping, and the DCT (version 2) was applied to each block individually. Finally, the values of the DCT coefficients were rounded, and the absolute value was calculated, as presented in Equation (1) and (2):

Where BDCTmk (u, v) represents the DCT of each 8 × 8 block and F (u, v) is the complete image of the absolute rounded value.

Derivative matrices

It is possible to use an edge detector to observe statistical changes in the image because the operation of placing one image over another (splicing) introduces statistical alterations on the image edges [³¹]. To detect them, the horizontal and vertical derivatives of the image were calculated using Equation (3) and (4):

Figure 1 General flow diagram of the proposed algorithm 

Where l(x-y) is the image, S _x and S _y are the image’s dimensions. Likewise, the horizontal and vertical derivatives of the frequency image are calculated using Equation (5) and (6):

Where S u and 𝑆 𝑣 are the image’s dimensions in frequency. It should be noted that up to this point, all resulting matrices had positive integer values.

Thresholding

To reduce the feature vector dimensions and the algorithm complexity, a threshold T must be performed. If the value of the derivative matrix is higher than T, this value is replaced by T. Conversely, if the value of the matrix is less than -T, this value is replaced by -T, as shown in Equation (7):

Where X(x, y) are all derivative matrices (E _h (x, y), E _v (x, y), F _h (u, v), F _v (u, v)). For this study, an adjustment was made in the range of thresholding based on the work presented by Kumar et al. [³⁵], who stated that better results were obtained when using a range of (i, j) =∈ {−T, −T + 2, . . . , T + 2, T }. Attention should also be given to the selection of an appropriate threshold, because a small T value may cause the transition probability matrix (TPM) not to be sensitive enough to detect alterations due to information loss. On the other hand, a high T value can cause the TPM to be extensive and therefore increase the feature vector and the complexity of the algorithm. In general, a threshold of T = 3 should be used to maintain a balance between sensitivity and complexity.

Probability transition matrix (TPM)

As previously mentioned, the splicing operation changes the correlation between the pixels of the original image. A random Markov process can be used to describe these correlation changes. In this case, the transition probability matrix was applied to each of the thresholded derivative matrices to characterize the Markov process. The horizontal and vertical transition probability matrices were calculated using Equation (8) and (9):

Where:

So, when applying (8) and (9) on horizontal and vertical derived matrices on space (Eh(x, y), Eh(x, y)) and the frequency (Fh(u, v), Fh(u, v)) a vector whose characteristics were 4 × (T + 1)2 was generated. igure 2 shows the TPM with a set threshold T = 4. Finally, the TMP became a data vector that fed a support vector machine (SVM) to be trained.

2.2 Copy-move detection

In case SVM has not detected any resampling alteration, the algorithm proceeds to perform copy-move detection. For this, the algorithm presented by Amerini et al. [³⁶] was used to detect key points with the SIFT algorithm, and the library VLFeat 0.9.21 [37] was employed.

Matching features through g2NN

Because two copied regions within the image have the same descriptors, the g2NN algorithm was used, which calculates the ratio between the Euclidean distance between a pair of candidate points and the next-nearest neighbor [³⁸]. If the condition presented in Equation (10) is met, a pair of matching points is created P = {p _i , . . . , p _n }; where each pi is each pair of (s _i , s _j ).

Where d _i is the Euclidean distance between the candidate point and the point to verify, d _i +1is the Euclidean distance between the candidate point and the next nearest neighbor, and τ is a defined threshold that allows the rejection of pairs with very different descriptors. If two or more pairs of matching points are not found, the algorithm identifies the image as authentic.

J-linkage grouping

To identify the areas where the alteration occurs, the J-Linkage cluster, also called Hierarchical Agglomerative Clustering, (HAC) was used [⁴⁵]. This procedure was performed with the pair of coinciding points coordinates but not with the Euclidean distance value. First, we proceeded with a random sampling of the pairs of coincident points p to generate m hypothesis of related transformations T = {T₁, . . . , T_m}. For each pair, a set of related transformations called ”preference set vector”: P S = {P S1 (pi) , . . . , P S _m (p _i )} where each P S _m (p _i ) was defined in Equation (11).

In other words, β is the distance between the T _m model and the pair of points p. If this value is less than 0.05, the pair of primary points and the pair of points of the copied region have similar transformations [³⁸]. The preference set vector is used in the HAC algorithm to find the transformations of both the original points and the copied points. After establishing the preference set vector, these were assigned to a cluster; then, for each iteration, each pair of clusters was merged with the smaller distance in space. The preference set vector of a cluster was calculated as the intersection of the preference sets of the matched pair, and the distance between the paired clusters was determined with the Jaccard distance (J _δ ) between the corresponding preference sets using Equation (12).

The sets without similarities have a value of 1, while similar sets have a value of 0. According to these parameters, Amerini et al. [³⁶] set a value of 1 as the cut-off grouping value. As a result, each cluster obtained at least one matching transformation among all its pairs. If more transformations are shared among all the elements of the cluster, they should be similar; for this reason, an estimation of the final transformation is determined by least-squares fitting. To dismiss outliers in cluster transformations, the fixed threshold N was used. Finally, if eight or more transformations (ind) are detected, or if there are four or more clusters (Cluster), the method alleges the image as falsified. Otherwise, it assumes that the image does not contain any modifications.

Figure 2 TPM result with a set threshold T = 4 

2.3 Algorithm adjustment

Given that the algorithm must identify different types of alterations, an adjustment was made for the final algorithm output. In the Markov preprocessing phase, an SVM was trained with authentic, resampling, and splicing images. If the SVM detected that the image had resampling, the algorithm indicated this. On the other hand, if the SVM identified the image as authentic or having splicing, the algorithm executed the copy-move detection process. If during this process, the copy-move block found the image to be free of copy-move alterations, the algorithm considered the image authentic. Table 1 presents the algorithm response depending on the SVM response and the copy-move algorithm. Because copy-move is a particular case of resampling, where there is no scaling or rotation, once the algorithm detects resampling, the output of the copy-move block is always resampling. Future work is expected to locate the region affected by copy-move or resampling.

3.1 Databases

In order to evaluate the algorithm, the most well-known forgery image evaluation datasets for splicing, resampling, and copy-move were selected. The datasets selected for evaluating splicing were CVLAB [⁴⁰], CASIA V1 [⁴¹], Columbia [⁴²] and Uncompressed Columbia [⁴³]. Ardizzone-dataset [⁴⁵], Coverage [⁴⁵], MIC-F2000 [³⁸] and CMFDdb-grip [⁴⁶] datasets were used for the evaluation of copy-move and resampling. A selection of datasets rich in formats, sizes, and quantities of images with different types of alteration were used. Table 2 shows the main characteristics of all the datasets that were used to evaluate the proposed algorithm, such as format, number of original images, number of altered images, and resolution. Taking into consideration all the datasets used for the study, a total of 7100 images were evaluated, of which 3666 were unaltered, 791 had resampling, 2213 had splicing, and 430 had copy-move alterations.

3.2 CVLAB forgery database

For this study, a new database called CVLAB forgery database was created and published in [⁴⁰]. This dataset contains 650 jpg format images that are 720 × 480. It contains 200 authentic images, 200 spliced images, and 250 copy-move images. The dataset has 50 images of animals, 40 of landscapes, 35 of flowers, 50 of buildings, 25 of people, and 50 of common objects. Figure 3 shows a sample of the images of the dataset.

3.3 Classifier performance

To demonstrate the detection performance, the classifier was evaluated by using the confusion matrix resulting in a 4 × 4 matrix that shows the classifier performance through the known values and predictions of the trained model. To determine the degree of reliability of the model, we calculated the accuracy (Acc), true positive rate (TPR), true negative rate (TNR), positive predictive value (PPV), the false positive rate (FPR), and the false-negative rate (FNR) using Equation [¹³-¹⁸] respectively. Alternative metrics that considered class imbalance were used, such as F1-score per class and weighted F1-score using Equation (19) and (20).

Where TP is the number of values predicted as positive that is indeed positive, TN is the number of values predicted as negative that is indeed negative, FP is the number of values predicted as positive that are negative, and FN is the number of values predicted as negative that are positive. 𝐹1−𝑠𝑐𝑜𝑟𝑒 𝑖 is 1 − score(i) is F1-score of each class, class _inst (i) are the total of instances of each class and total inst are the total of instances used for classification.

3.4 Threshold T selection

As mentioned before, the selection of a threshold T is essential since this allows adjusting the robustness of the method together with the computational cost. At a higher value of T, a better sensitivity with a higher computational cost is obtained, and, conversely, a lower value of T loses sensitivity with a lower computational cost. Therefore, to identify the selection of the ideal threshold T, the splicing and resampling datasets mentioned above were trained with values of T equal to 4, 6, 8, and 10. Subsequently, the accuracy and precision of each model were estimated. All evaluations were performed with cross-validation of 10 folds. Finally, with the best value of the threshold T obtained, a model with an SVM with a Radial Basis Function (RBF) kernel was trained as a classifier to identify splicing and resampling alterations with the CASIA V1, Columbia, Columbia Uncompressed, CVLAB, and MICC-F2000 for a total of 7100 images. Only 161 random images of the Ardizzone-dataset were evaluated because this dataset only has 50 original images; therefore, imbalanced classes may affect the results of the evaluation.

4.1 Detection performance for SVM model

Table 3 shows the results of the evaluation carried out for the Resampling and Splicing detection with different T thresholds with multiple databases.

As is evidenced in Table 3, a threshold of T = 4 obtained an accuracy of 90%, while T = 10 obtained an accuracy of 89%.

4.2 Performance of the proposed method

The confusion matrix that presents the result of evaluating all the datasets with the proposed methodology for the identification of splicing, resampling, and copy-move is presented in Figura 4. For this evaluation, a trained model with an SVM was selected with an RBF kernel with a threshold of T = 4 since it obtained an outstanding result with a lower computational cost. The confusion matrix, shown below, was adjusted with the following color code: green represents true positives, purple represents false authentic positives, red represents false splicing positives, blue represents false resampling positives, and orange represents false copy-move positives.

Figure 3 CVLAB Forgery Database images 

Table 2 Databases 

Table 3 Results of SVM model with different T value 

As presented in Figure 4, the proposed method obtained a general accuracy of 91% and weighted-F1 of 92%. Where its true positive rate for authentic images was 91%, 89% for images altered by splicing, 99% for images altered by resampling, and 79% for images altered by copy-move, Table 4 illustrates the results of the algorithm in greater detail.

The high TPR values for each of the classes indicate that the algorithm is effective at identifying each type of alteration, with resampling identification being the highest with 0.99 and copy-move being the lowest with 0.79. The highest PPV value was 0.98, while the lowest accuracy was copy-move with 0.46. On the other hand, the highest accuracy corresponded to the resampling class with 0.99 and the lowest to authentic images with 0.91.

Table 4 Performance of the proposed method 

A comparison of our proposed algorithm with other techniques for the identification of various types of alterations [⁴⁷-⁴⁹] is presented in Table 5.

Table 5 shows a 97% and 99% overall accuracy for the algorithm proposed by Sharma and Hema [⁴⁸,⁴⁹], respectively, while the proposed algorithm’s accuracy in this study is 91%. It is important to note that Prakash [⁴⁷] only used three, Sharma, and Ghanekar [⁴⁸] four, Hema Rajni [⁴⁹] three, and the proposed technique eight datasets.

Figure 4 Confusion matrix of the proposed method