In this work, through thorough statistical analysis of unmodified commodity devices, we have found an optimum feature that improves the accuracy of RF-PUF significantly on a suite of commodity hardware devices leading to > 99.8% accuracy, along with PUF property analysis and security vulnerability analysis. Detailed contributions are as follows:

(1) Feature engineering: Principal component analysis has been performed on the existing feature set found in the literature to find the dominant feature. Through moment analysis on the dominant feature (i.e. carrier frequency offset) we demonstrate that the addition of a feature called COV (ratio of standard deviation and mean of carrier frequency offset) significantly helps in achieving high ( > 99.8 % ) accuracy (Section 4.3).

The rest of the paper is structured as follows: Section 2 provides relevant works on RF fingerprinting and device authentication. Section 3 provides an overview of our experimental setup, data collection, and data processing method. Section 4 presents a new feature set development using statistical analysis and corresponding performance enhancement. Section 5 explores the design space in detail. Section 6 analyzes various PUF properties in the context of RF-PUF. Section 7 discusses possible attack models and the resilience of RF-PUF against them. Finally, section 8 concludes this paper.

For experimental validation, 30 XBee S2C modules are chosen (IEEE 802.15.4 standard) which is designed for industrial and commercial use. Figure 2A shows the Xbee devices whereas Figure 2B, and Figure 2C show the block diagram and the actual setup. The TX and RX are kept 1 m apart. Using SMA cable, a HackRF One software-defined radio (SDR) module has been connected either to the TX (case 1) or to the RX (case 2) to extract data excluding (case 1) or including (case 2) wireless channel.

A 31-bit pseudo-random bit sequence (PRBS) is generated in MATLAB and fed to each TX which transmits this data for 60 s with QPSK modulation at 2.465 GHz and 230,400 bps baud rate. These data were captured in a Xbee RX module. Simultaneously, data were also captured by a HackRF one software-defined radio (SDR) module, sampled at 6 MSps, and stored by GNU Radio. The captured data are divided into several frames, each containing a number of samples. From the constellation diagram of the frame data (Figure 3), it is found that some frames have no significant data points and contain only noise as the Xbee devices transmit data intermittently due to their buffer limitation. These blank frames containing only noise were discarded.

This dataset contains raw data collected from 30 Xbee S2C transmitters for both cases (excluding and including the channel) in binary format. The total size of the dataset is 155.4 GB (each transmitter data is ∼2.5 GB). It can be downloaded from Sparclab RF-PUF Dataset (Bari and Sen 2022).

In our work, CFO and I-Q data are taken as features just as in the original RF-PUF paper (Chatterjee et al., 2019). The previously generated frames are filtered using matched filtering, frequency compensated (both fine and coarse), and finally synchronized using timing recovery. In this process, CFO is found as a byproduct. Along with CFO, the compensated in-phase and quadrature-phase components in four quadrants are used as features. The 9 features (CFO + 4 I-components + 4 Q-components) from each frame and 1,000 frames from each TX lead to a feature set of 9 × 1000. The final feature matrix is a combination of these feature sets from all 30 devices and has a size of 9 × 30000.

The whole feature data are divided into 70%, 15%, and 15% respectively for training, validation, and test purposes and fed into a neural network (NN). The performance of the neural network is tested by varying the number of neurons and hidden layers. Figure 4A shows the accuracy of the trained model for different neural networks. The accuracy is less than 75% in all test cases. Since exploring different NN configurations does not provide expected accuracy, our choice here is to: 1) form an improved feature set to be used with the NN 2) use different machine learning (ML) algorithms 3) use more data. We first search for an improved feature set for better accuracy. Later, the effect of more data is shown in subsection 5.1, 5.2 and a comparison of different ML algorithms and NN is discussed in subsection 5.4.

COV is included as the 10^th feature in our existing feature matrix. From PCA analysis, it is already revealed that the I-Q features contribute to much fewer variances and can be discarded by trading some accuracy. Since our goal is to achieve maximum possible accuracy, we still keep them as features. Also, I-Q values contain channel information, which will help the NN to compensate for the wireless channel (channel effect is explained in subsection 5.5).

After including COV as the 10^th feature, our neural network was trained, validated, and tested again with the new feature matrix. Figure 4D shows that the performance of the network has improved drastically. With just a single hidden layer, > 95 % accuracy can be achieved using 10 neurons and can hit up to 99.9% accuracy by increasing the number of neurons.

Figure 5A shows the plot of detection accuracy versus the number of samples in each frame for different neural networks. The general trend (Bold red arrow) is that, for each NN configuration, detection accuracy improves with the increase in the number of samples (along the x-axis). This is expected because a higher number of samples provide more information and hence better performance. We want to mention that there might be some temporary sporadic drops in accuracy (as in the red line where accuracy slightly drops from 50 to 100 samples), but that does not represent the general trend which clearly shows that more samples translate to better accuracy. Also, > 95 % accuracy point is reached at around 150 samples per frame which is equivalent to 12.5 ms of total data (or 1.8 ms test data). Hence, we can reach the 95% accuracy bar using quite small test data.

Figure 5B shows accuracy versus the number of neurons per layer for two different frame numbers, 500 and 1000. With a higher frame number, the information content of each transmitter device increases. As the NN gets more information about the device, its performance improves and the detection accuracy gets better as shown by the blue (1000 frames) and red lines (500 frames) respectively. We can generalize the previous subsection (sample number effect) and this subsection as this: more data render better performance.

Figure 5C shows the plots of accuracy versus the number of neurons in each hidden layer. As the number of neurons increases along the x-axis, accuracy, in general, gets better (there might be sporadic peaks or drops as in the case of the blue line with an outlier peak at 30 neurons, but this does not represent a general trend). Also, as the number of hidden layers increases, the network performs better initially (from blue to red line), but later it creates an overfitting problem (the green line) where the model capacity is too large compared to data. This phenomenon directly manifests itself as a degradation in performance. Hence, there is an optimum model capacity up to which accuracy increases, and beyond that accuracy drops.

It has been observed that the COV values vary significantly among different transmitters. When a simple feature displays a significant separation among different classes, it can be modeled with a complex if-else ladder structure. This implies that even simple ML algorithms (e.g. Tree) can show good results. Figure 6A shows that some popular ML algorithm achieves > 95 % accuracy.

The true power of the neural network comes into play when the number of TX increases as shown in Figure 6B. For this, features are generated for 300 TX devices following a Gaussian distribution (as in (Chatterjee et al., 2019)) with the same mean and variance as that of the original 30 TX devices, for both inter and intra-class variations. Figure 6B shows that as the number of TX increases, accuracy falls after a certain point ( ∼ 100 TX) even for support vector machines (SVM), and it fails to converge for > 150 TX.

So far, nonidealities due to TX were considered and the wireless channel was ignored (TX and RX are connected via SMA cable). But the channel itself adds some nonidealities. Here, the effect of a static wireless channel (1 m of fixed TX-RX separation) has been analyzed. Figure 7 shows accuracy versus neuron number in a single layer, with and without the wireless channel. For iso-accuracy of 95%, wireless channel demands slightly higher model capacity (10 vs. 15 neurons). But when the number of neurons increases ( > 50 ) , both curves merge and render similar accuracy.

In one of our recent works (Bari et al., 2021b), we applied RF-PUF on the ORACLE dataset which contains data for 16 USRP X310 TX for both static and quasi-static (variable TX-RX separation) cases with a channel length varying from 2 to 62 ft. We have shown that RF-PUF achieves 100% accuracy up to 38 ft and > 95 % accuracy even at 62 ft channel length. This result confirms that the RF-PUF approach can make the channel compensation with the help of NN and render high performance even in a long wireless channel. On a side note, that work combined with current work, also confirms that RF-PUF achieves high accuracy on experimental data in different platforms (XBee vs. USRP radios using WiFi) for different devices.

RF-PUF does not add any additional circuitry on the TX side. Hence there is no extra computational burden at the TX end. On the receiver side, it employs just a multilayer perceptron (MLP) or NN along with the standard receiver. The standard receiver corrects the received signal and in the process discards various system-level nonidealities which are used as features to the NN. Hence, the computational complexity of RF-PUF is that of a neural network. For an n-dimensional input (n = 10 for our case), the training phase (done only once) has a computational complexity on the order of O ( n 5 ) , whereas the inference phase has a computational complexity on the order of O ( n 4 ) (a derivation of the orders can be found at (Fredenslund 2022)). Also, the statistical feature formation requires mean and standard deviation calculation which has computational complexity in the order of O ( n 2 ) , which is negligible compared to the inference order.

A PUF class P is constructible if we can create a new PUF instance p u f m ∈ P through a process, P .Create: p u f m ← P .Create , where puf _m has entropy that makes it distinct from other PUF instances puf _n,n≠m . In the case of RF-PUF, the source of entropy is the manufacturing process variation. During fabrication of ICs, we have within die and die-to-die variation which is due to the limitation of the manufacturing process. In contrast to many other PUF classes where we need a separate mechanism for PUF instance creation, the manufacturing process of the integrated circuit itself serves as the creation process for RF-PUF which is one of its advantages.

A PUF class P is evaluable if for a random PUF instance p u f m ∈ P and a random challenge (x), we can evaluate a response y: y ← puf _m (x). In our case, the challenge is a randomly generated bitstream in MATLAB that is fed into the transmitter and the corresponding response is the analog signal that contains the unique physical signature of the transmitter.

Uniqueness refers to how different each instance of a PUF class P is from each other. A measurement metric that is used to represent PUF uniqueness is called inter-PUF hamming distance and is defined as: H D i n t e r ≅ d i s t a n c e Y m α x , Y n α x

Here, Y m α ( x ) and Y n α ( x ) are the responses from puf _m and puf _n (two instances of PUF class P ) under the same environmental condition α and same challenge x. Ideally, these inter-chip hamming distances should be much greater than any intra-chip hamming distances to distinguish them separately. In our experiment, our PUF class P = RF-PUF and puf _i , (where i = 1, 2, …, 30) are the instances of that class (30 Xbee devices).

To calculate HD _inter , the first 1000 frames from each of the transmitters are taken. Each frame contains 3600 samples. Our features remain unchanged: CFO, eight I-Q component values, and COV. But after taking 10 features from each of 1000 frames, instead of using them as a feature matrix for each transmitter, the mean values of the features are taken across all the frames. This means that instead of representing each transmitter as a 10 × 1000 feature matrix, it is represented as a 10 × 1 feature vector. The reason for taking the average value across the frames is that the frames have an associated timestamp with them i.e., each frame data are collected from time to time. So, each frame faces slightly different environmental conditions such as heating of the transmitter due to data transmission for a long time, external interference, noise, etc. Averaging the feature values across a large number of frames mitigates the environmental factors, especially noise. Also, taking the first 1000 frames from each transmitter ensures the same initial heating pattern across devices. So the final outcome is that the feature vector for each transmitter has a very similar environmental factor α, which is one of the conditions of inter-chip hamming distance calculation.

After taking the feature vector from each transmitter, the Euclidean distance was calculated in ten-dimensional feature space as hamming distance. For puf _m , let us denote CFO _m = carrier frequency offset, COV _m = coefficient of frequency offset variation, I _k,m = in-phase component in the k ^th quadrant, and Q _k,m = quadrature-phase component in the k ^th quadrant. Then distance d _m,n between puf _m and puf _n instances is given by: d m , n 2 = C F O m − C F O n 2 + C O V m − C O V n 2 + ∑ k = 1 4 I k , m − I k , n 2 + ∑ k = 1 4 Q k , m − Q k , n 2 (1)

The inter-chip distances were calculated for each transmitter with respect to all 30 transmitters (including the chip under test), which leads to a 30 × 30 symmetric matrix (upper and lower triangular matrices with same values since d _m,n = d _n,m = inter-chip distance between puf _m and puf _n ) with a principal diagonal of zeros (self-distance). It is found that the worst-case scenario with minimum distance, HD _inter,min = 0.2307, and the best-case scenario with maximum distance, HD _inter,max = 10.149.

In literature, often a mean inter-puf distance, μ _inter , is reported which is the average of all HD _inter . The formula is: μ i n t e r = H D i n t e r ̄ = 2 N p u f × N p u f − 1 × N c h a l ∑ H D i n t e r Where N _puf is the number of puf instances (N _puf = 30 for us), and N _chal is the number of challenges (N _chal = 1, since we are not varying our challenge). Using this formula, we find that μ _inter = 3.703.

Figure 8C shows the probability mass function distribution of 435 ( = 30 × 29 2 ) inter-PUF distances. The density function is right-skewed, that’s why Weibull fitting (which is exponential in nature) fits it more accurately than normal distribution fitting. This fitting shows that on the right side the curve is more sparse but on the left side it is more centered instead of being sparse, which is good because that will ensure that the inter-PUF values don’t go to overlap intra-PUF distances which should ideally be at zero.

PUF responses are in general dependent on various environmental factors that render any PUF instance response as a probabilistic function. This means that a particular PUF instance can provide slightly different values of features based on varying environmental conditions. For authentication purposes, this poses an issue. Reliability refers to how resilient a PUF instance is against environmental factors e.g. noise, interference, temperature, supply voltage, etc.

A measurement metric that is used to represent how reliable a particular instance of a PUF class P is intra-puf hamming distance and is defined as: H D i n t r a ≅ d i s t a n c e Y m α x , Y m β x Here, Y m α ( x ) and Y m β ( x ) are the responses from puf _m under two distinct environmental conditions α and β and same challenge x. Many HD _intra distances are calculated at different environmental conditions. Ideally, these intra-chip hamming distances should be zero.

To calculate HD _intra , we follow two steps. Let us consider one particular PUF instance puf _m . In step 1, the first 1000 frames (frame number 1 to frame number 1000) were taken from puf _m , each frame containing 3600 samples. Then mean values of the previously mentioned ten features were taken just as before to represent it as a 10 × 1 feature vector. Let us represent this vector as f _v,1. Then in step 2, the first 5 frames are skipped and the next 1000 frames are taken from frame number = 6 to frame number = 1005. Step 1 is repeated here to get the next feature vector f _v,2. Then next 1000 frames are taken from frame number = 11 to frame number = 1010 and a feature vector f _v,3 is formed. This process is repeated 80 times to form 80 different feature vectors f _v,α ; α = 1, 2, … , 80. These 10 × 1 feature vectors are stacked together to form a feature vector set f _set,m of size 10 × 80 for puf _m . The whole process is then repeated for all 30 devices.

The purpose of taking frame-shifted or time-shifted frame groups is to consider the time factor. Each frame has a duration of 0.6 ms, so 5 frames gap in between two frame groups renders a time difference of at least 3 ms (in reality the difference is much larger since the transmitter transmits data for a small time and most of the frames are just noise which are filtered in data pre-processing step). The 80 time-spaced frames, in reality, cover almost half a minute. Our 2.4 GHz clock will have LO drift cycle time in the nanoseconds range. Hence, half-minute data can incorporate significant environmental factors into frame data. So, it can be assumed that the feature vectors f _v,α ; α = 1, 2, … , 80 in feature vector set f _set,m of puf _m represents α = 80 different environmental conditions.

Now, for each instance puf _m , Euclidean distance is calculated in 10-dimensional feature space among the feature vectors in the feature vector set using Eq. 1. This results in a symmetric matrix of size 80 × 80 with a principal diagonal of zeros. This process is repeated for other transmitters as well. Essentially it gives us 30 matrices of size 80 × 80 for intra-PUF distances. In the best-case scenario, the minimum distance is HD _intra,min = 7.23 × 10^–5 and in the worst case scenario, the maximum distance is HD _intra,max = 0.73.

Figure 8A shows the probability mass function distribution of 94800   ( = 30 × 80 × 79 2 ) intra-PUF distances. The density function is right-skewed and Weibull distribution gives better fitting for it just like inter-PUF cases. This fitting shows that on the left side the curve is strongly centered towards zero, but has a diminishing trail on the right. this trail goes on to overlap inter-puf distances slightly and causes a few detection errors. Detection probability is discussed in the next subsection.

Finally, a mean intra-PUF distance, μ _intra , is calculated which is the average of all HD _intra . The formula is: μ i n t r a = H D i n t r a ̄ = 2 N p u f × N c h a l × α × α − 1 ∑ H D i n t r a Where N _puf is the number of puf instances (N _puf = 30 for us), N _chal is the number of challenges (N _chal = 1, since we are not varying our challenge) and α is the number of environmental conditions (α = 80 in our study). Using this formula, it is found that μ _intra = 0.136.

In the previous two subsections, both inter-PUF and intra-PUF hamming distances and their mean values: μ _inter = 3.703 and μ _intra = 0.136 are calculated. Their comparison shows that μ _inter > μ _intra , which establishes that on average the PUF instances can be distinguished from each other. But the mean value does not depict the full story. Figure 8B shows the fitted distribution curves superimposed on each other. The brown curve (intra-PUF distribution) is skewed to the left and the blue curve (inter-PUF distribution) is skewed to the right and they mostly cover different regions. However, there is slight overlapping between them which is shown in the inset as a zoomed version of the overlapping area. Ideally, there should be no overlapping. But in a practical scenario, this overlapping region is the source of detection error.

From the definition of identifiability, a PUF class P is identifiable if it is reliable as well as unique, and if the probability of inter-PUF variation being greater than intra-PUF variation is very high. Mathematically: P r o b a b i l i t y H D i n t e r > H D i n t r a ≈ 1

In the previous two subsections, 94800   ( = 30 × 80 × 79 2 ) intra-PUF distances and 435   = 30 × 29 2 inter-PUF distances have been calculated. Now, each of these inter-puf distances is compared with each of the intra-PUF distances that leads us to 435 × 94800 = 41238000 cases, among which, HD _inter > HD _intra is found in 41184206 cases. P r o b a b i l i t y H D i n t e r > H D i n t r a = 0.9987

This is a very high probability and close to 1. This proves that RF-PUF has strong identifiability and this property along with reliability, uniqueness, constructability, and evaluability manifests RF-PUF as a distinct PUF class. This is the first-ever experimental validation of RF-PUF as a distinct and strong PUF class by itself.