1. Introduction

The Durand Adaptive Psychopathic Traits Questionnaire (^{DAPTQ; Durand, 2019c}) is a 38item selfreport in strument developed to examine adaptive traits associ ated with psychopathic traits. The DAPTQ uses a total score, as well as eight subscale scores: Leadership (4 items), Logical Thinking (5 items), Composure (6 items), Creativity (4 items), Fearlessness (6 items), Focus (4 items), Extroversion (6 items), and Management (3 items). Three of the DAPTQ’s factors mirror factors observed in another instrument in the field of psycho pathic traits Fearlessness, Extroversion, and Compo sure are strongly associated with the factors Fearless ness (r = .59), Social Potency (r = .77), and Stress Immunity (r = .61) from the Psychopathic Personality Inventory (PPI; ^{Lilienfeld & Andrews, 1996}), as it was observed in a previous study (Durand, 2019c). The DAPTQ’s other factors, although not included in any measure of psychopathic traits, have been associated with various psychopathic traits across several studies (for a review, see ^{Durand, 2019b}). It should be noted that the DAPTQ’s framework is based on the theory that adaptive traits play a central role in the construction of psychopathic traits. This theory is part of an ongoing debate, as traits considered adaptive do not generally correlate with externalizing behaviors, which are considered primordial within the concept of psychopathy (^{Crowe et al., 2021}; ^{Lilienfeld et al., 2012}; ^{Miller & Lynam, 2012}). ^{Durand (2019c)} also emphasizes that the DAPTQ is not a measure of psychopathy, nor of psychopathic traits, as the instrument does not cover any maladaptive traits.

While the DAPTQ is showing promising psychome tric properties (^{Bronchain et al., 2021}; ^{Bronchain et al., 2020}; ^{Durand, 2019a}), it does not include a validity scale, which can compromise the quality of the results obtained by the instrument. Among other factors, careless responding can negatively impact results by de creasing statistical power and estimates of internal consistency, as well as by affecting correlations between variables (^{Curran, 2016}; ^{Huang et al., 2015}). Although it remains unclear which populations are most susceptible to careless responding, estimates suggest that 812% of respondents from a typical study answer carelessly or with insufficient effort (^{Curran, 2016}). Identifying these individuals is particularly important in personality related research, as previous findings suggest that careless respondents typically exhibit lower levels of agreeableness, conscientiousness, and openness (^{Berry et al., 2019}).

Due to the lack of a validation scale, many of the previous studies using the DAPTQ had to rely on validity scales included in other instruments administered at the same time as the DAPTQ. For instance, a valida tion study of the DAPTQ used the Variable Response Inconsistency (VRIN) of the Psychopathic Personality Inventory Short Form (PPISF), which measures the extent to which item pairs with similar content are an swered inconsistently (^{Durand, 2019a}). These types of inconsistency scales are developed by identifying highly correlated item pairs. A total score is calculated based on the sum of the absolute value of the difference for each pair of items. Lower scores reflect smaller discrepancy between similar items, which should, theoretically, be endorsed in a similar manner. As highlighted by ^{Mowle and colleagues (2017)}, the performance of inconsistency scales should be crossvalidated in multiple samples to ensure generalizability, and it should also be validated against randomly generated protocols.

Based on the issues associated with careless and random responding, it would be beneficial to develop a validation scale specific to the DAPTQ. The DAPTQ could then be used in studies without any other self reported instrument. The present study has four objectives. The first objective is to develop an inconsistency responding scale for the DAPTQ (IRSDAPTQ) using the archival data from ^{Durand’s (2019a)} study #1 and study #2. The second objective is to examine the external and criterion validity of the newly developed IRS DAPTQ by examining associations with other inconsistency responding scales, as well as associations with measures of personality traits, such as conscientiousness, that have shown a negative association with careless responding (^{Kelley et al., 2016}). The third objective is to examine the internal validity of the scale by comparing IRSDAPTQ scores obtained from individuals (students and the general population) who completed the DAPTQ with randomly generated DAPTQ scores. The last objective is to identify a tentative cutoff score to classify potentially invalid profiles based on comparisons of respondent data with a randomly generated sample of N = 1000 DAPTQ protocols.

2. Method

The present study used the combined dataset of both samples from ^{Durand (2019a)}, as well as two original samples and one generated sample. While Durand’s two original studies (2019a) used the 41item version of the DAPTQ, the three items related to the Money Smart subscale were not used here because Durand had found that the items included in the Money Smart factor were derived from multiple constructs unrelated to financial management. Across all samples, the 38 items were answered on a 6point scale: Strongly Disagree, Disagree, Slightly Disagree, Slightly Agree, Agree, Strongly Agree. There was no missing data in any of the datasets. For all studies, the requirements were to be over 18 years old and to be fluent in English. Informed consent was obtained from all participants.

3. Results

A Pearson correlation among all 38 items of the DAPTQ was performed to identify strong correlations (r .50) within Sample 1. In addition to a minimum correlation of .50, no individual item could be included in more than one pair (i.e., items were included only in the pair with the strongest correlation). Out of the 46 pairs of items with a correlation of over .50, 14 pairs of unique items were selected. Selection of the pairs was done by ranking the results of the correlations in decreasing order, starting with the strongest correlation (pair 619, r = .78), and then proceeding to include the next strongest pair, while excluding pairs containing an item from a pair previously selected. Based on the mean difference for each pair, two pairs were excluded for having a mean difference ≥1.00. This threshold of 1.00 to exclude pair of items is in line with previous research related to the development of a response inconsistency scale (^{Mowle et al., 2017}). The total score of the IRSDAPTQ was then calculated by summing the absolute value of the difference between each pair of items (range: 060). Table 1 summa rizes the item correlations and mean differences for each pair by sample. The final version of the IRSDAPTQ con sists of 12 pairs and includes at least one pair from each scale of the DAPTQ. The final scale is as follows:

Subsequently, external validity was examined by com paring the results of the IRSDAPTQ with the VRIN of the PPISF from Durand’s study #1 and with the VRIN of the PPISF and the TAPIR of the TriPM from study #2 (2019a). For study #1, the IRSDAPTQ was weakly correlated with the VRIN (r = .19, p = .002). For study #2, the IRSDAPTQ was weakly correlated with the TAPIR (r = .13, p = .031) and was not correlated with the VRIN (r =.07, p = .267). The TAPIR and the VRIN were also weakly correlated with each other (r = .23, p < .001). Validity was further explored by examining the association between the IRSDAPTQ and the six HEXACOPIR60 factors of Sample 1 through a Pearson correlation. Based on a Bonferroni correction of .05/6 = .008 to correct for multiple testing, only one significant result emerged: a weak negative correlation between the IRSDAPTQ and conscientiousness (r = .21, p = .001). The VRIN of the PPISF of study #1 was not significantly correlated with any of the HEX ACO’s factors.

Internal validity of the IRSDAPTQ was assessed by comparing the means of the IRSDAPTQ for samples of students and the general population with randomly generated DAPTQ protocols (N = 1000). These random protocols were generated via an excel spreadsheet with a command to generate a whole number between one and six, inclusively, for each item of the DAPTQ, repeated 1000 times. The responses were equally weighted. While 100% random protocols may not be an accurate reflec tion of careless responding for some participants, they accurately represent individuals who complete question naires randomly in order to quickly finish the task and receive the compensation (e.g., course points). Figure 1 displays the distribution of DAPTQ scores for the derivation, undergraduate, general population, and ran dom samples. The IRSDAPTQ scores for each human sample were consistently lower (derivation: M = 8.35, SD = 3.34; undergraduate: M = 8.88, SD = 3.59; gen eral population: M = 8.10, SD = 3.33) than those for the randomly generated sample (M = 21.32, SD = 4.87; see Table 2 for the test results).

Lastly, a receiver operating characteristic (ROC) curve was computed to find a possible cut score to identify po tentially problematic protocols with high levels of inconsistencies. The ROC curve is a graphical plot displaying the ability of a binary system to classify data based on sensitivity (i.e., correctly detecting positive cases) and specificity (i.e., correctly detecting negative cases). Table 3 shows the classification accuracy of potential cut scores for differentiating genuine data from randomly generated DAPTQ protocols. Based on these results, a cutoff of 14 might be the most appropriate, yielding an adequate tradeoff between sensitivity and specificity.

4. Discussion

The purpose of the present study was to develop an in consistent responding scale, the IRSDAPTQ. An initial scale of 14 item pairs was developed from the derivation sample. Based on mean differences between the items in each pair, two pairs were excluded, yielding a scale of 12 item pairs covering all eight subscales of the DAPTQ. The validity of the IRSDAPTQ was assessed by com paring the scores with scores generated from alterna tive inconsistency scales and with a measure of personality traits, as well as by comparing mean differences on the IRSDAPTQ between protocols completed by participants and randomly generated protocols. Across all samples, the IRSDAPTQ strongly differentiated genuine data from randomly generated protocols.

Figure 1 Distribution of the IRSDAPTQ Scores for the Derivation, Undergraduate, General Population, and Randomly Generated Samples 

The results of the Pearson correlations between the IRSDAPTQ and both the VRIN and the TAPIR may be explained by the use of different scales. The DAPTQ uses a 6point rating scale, while both the PPI and the TriPM use a 4point rating scale. This may increase the variability of responses in the DAPTQ, hence decreasing its correlation with the two inconsistency scales. Interestingly, the two inconsistency scales did not correlate strongly with one another either. It is possible that a correlation can only emerge in datasets including high numbers of outliers.

In line with previous findings, the inconsistency scale of the DAPTQ was significantly negatively correlated with conscientiousness. Similar correlations were observed between conscientiousness and the inconsistency scale of the Triarchic Psychopathy Measure (r = .26, p < .01; ^{Mowle et al., 2017}) and the inconsistency scale of the Psychopathic Personality Inventory (r = .18 to .23, p < .01; ^{Kelley et al., 2016}). These findings sup port the hypothesized relationship between inattentive ness when completing questionnaires and certain basic personality traits, especially conscientiousness (^{Bowling et al., 2016}).

The sensitivity and specificity associated with the proposed cut score is in line with previous findings. For instance, ^{Mowle and colleagues (2017)} suggested a cut score between 11 to 13 for their inconsistency scale, which corresponds to a sensitivity of 96.8 to 87.4% and an un dergraduate specificity of 90.1 to 95.5%. This specificity is also in line with the inconsistent responding usually observed in undergraduate students (^{Curran, 2016}). It should be noted that for the proposed cut score, the specificity of the general population sample recruited on Reddit was substantially higher (93.4%) than the speci ficity of the undergraduate population (90.4%).

5. Conclusion

While the validity of the IRSDAPTQ cannot be confidently established due to the small number of samples investigated in the current study, the scale successfully identified random profiles and showed similar sensitivity and specificity across human samples. There are, however, a few notable limitations to the present study. First, the IRSDAPTQ could not be checked against a well validated inconsistency scale for the new samples, without which validity cannot be established (^{Bagby & Sellbom, 2018}). A subsequent study should examine the scores obtained on the IRSDAPTQ in correlation with validity scales assessing response inconsistency. Second, the present study uses a 100% randomly generated sample, which mimics individuals filling in random answers to complete a study as quickly as possible, rather than individuals who answer carelessly only on occasion throughout the questionnaire. A future study should instruct participants to complete the instrument without reading every statement in order to mimic occasional disinterest in the questions. In addition to addressing these limitations, future research should further investigate the effect of offering course point/monetary rewards to undergraduate students by comparing their performance to that of uncompensated participants from the general population, since rewards could also affect inconsistent responding. Additional samples, such as employee groups, should also be re cruited to examine the validity of the IRSDAPTQ across a wider range of populations.