In this section, we describe the calculation method of the (ε,τ) entropy and sample entropy. We prepare a chaotic temporal waveform of laser intensity generated by a semiconductor laser with optical feedback (Uchida, 2012). The temporal waveform is sampled at a sampling interval τ, as shown in Figure 1. The discrete time t is represented as iτ (i = 1,2, …, N, where Nτ is the total length of the temporal waveform). The amplitude of the temporal waveform is quantized at a quantization interval ε. Next, we randomly select a sampling point x i τ in the temporal waveform, which is called the reference data, and prepare a reference vector x i d with vector length d. The reference vector x i d is expressed as follows: x i d = x i τ , x i + 1 τ , … , x i + d − 1 τ (1)

The distance between two vectors x i d and x j d ( i ≠ j ) is defined by calculating the maximum value of the difference in each vector component as follows. d i s t [ x i d , x j d ] = ⁡ max x i τ − x j τ , x i + 1 τ − x j + 1 τ , … , x i + d − 1 τ − x j + d − 1 τ (2)

We calculate the probability A i d of the existence of the neighboring vectors within the distance ε as follows. A i d = N u m b e r x i d ;   d i s t [ x i d , x j d ] ≤ ε N u m b e r x i d , (3) where N u m b e r [ x i d ;   d i s t [ x i d , x j d ] ≤ ε ] represents the number of the neighboring vectors within the distance ε and N u m b e r [ x i d ] indicates the total number of the vectors x i d . The value of A i d is calculated for different sampling points i. The average of the base 2 (binary) logarithm of A i d is calculated for the (ε,τ) entropy as follows. A d = − 1 R ∑ i = 1 R log 2 ⁡ A i d     for  ε , τ  entropy , (4) where R denotes the number of reference data points. By contrast, for the sample entropy, the average of A i d is calculated first, and the logarithm of the average value is calculated as follows: A d = − ⁡ log 2 1 R ∑ i = 1 R A i d     for sample entropy (5)

The entropy E is obtained from the difference between A d and A d + 1 as follows. E = A d − A d + 1   bit / sample (6)

The entropy rate is obtained by dividing the entropy E by τ as follows. h = E τ   bit / second (7)

The entropy rate relies on both the quantization and sampling intervals (ε and τ) of chaotic temporal waveforms and the vector length d. It has been known that the (ε,τ) entropy converges to the KS entropy under the limit of ε → 0 and τ → 0 (Cohen and Procaccia, 1985; Gaspard and Wang, 1993).

For the (ε,τ) entropy, when ε is set to be too small, A i d can be zero, and Eq. 4 diverges to negative infinity. To avoid this issue, the reference vector is considered as the neighboring vector, so that N u m b e r [ x i d ;   d i s t [ x i d , x j d ] ≤ ε ] is one or larger. By contrast, this modification is not required for sample entropy because the average of A i d is first calculated, and the logarithm of the averaged probability is obtained. Therefore, the sample entropy is expected to be more reliable for a large d.

Figure 2 shows the experimental setup for generating chaotic temporal waveforms in a semiconductor laser with time-delayed optical feedback. A distributed-feedback (DFB) semiconductor laser (NTT Electronics, KELD1C5GAAA) was used as a light source. The injection current of the semiconductor laser was set to 38.5 mA (3.5 times the lasing threshold). A fiber reflector was used to produce time-delayed optical feedback for the laser to generate chaotic laser outputs. The feedback delay time was set to 25.0 ns. The feedback power is set to 58 μW, which corresponds to the feedback ratio of 0.008 compared with the laser power. The chaotic temporal dynamics of the laser intensity can be obtained using time-delayed optical feedback. The chaotic temporal waveforms were converted into electric signals using a photoreceiver (Newport, 1554-B, 12 GHz bandwidth). The converted electric signals were measured using a digital oscilloscope (Tektronix, DPO73304D, 33 GHz bandwidth, maximum sampling rate of 100 GigaSample/s (GS/s), and 8-bit vertical resolution). The sampled data were stored on a digital oscilloscope and were used to estimate the entropy rate. The radio frequency (RF) spectra of the electric signals were observed using an RF spectrum analyzer (Agilent Technologies, N9010A-544, 44 GHz bandwidth).

Figure 3 shows the experimental results for the chaotic temporal waveform of the laser intensity, the corresponding RF spectrum, and a histogram of the amplitude of the chaotic temporal waveform. Chaotic temporal waveforms are obtained and broad spectral components are observed in the RF spectrum. In addition, the histogram of the chaotic temporal waveforms appears to be asymmetric, as shown in Figure 3C. This asymmetry can be eliminated by subtracting the original chaotic signal from its time-delayed signal (Reidler et al., 2009) to improve the statistical characteristics of the generated random bits. However, our goal is to estimate the entropy rate of the original chaotic signal, and we use the chaotic data with the asymmetric histogram shown in Figure 3C. The chaotic temporal waveforms are acquired so that their standard deviation is set to σ = 32 in the 8-bit vertical resolution with a range of [-128, 127], where the histogram is ranged within ±4σ.

We estimate the (ε,τ) entropy and sample entropy of the experimentally obtained chaotic temporal waveforms in Figure 3. The parameter values for the entropy estimation are set as follows: The sampling interval of τ = 100 ps (10 GHz in frequency), the quantization resolution of ε = 2^8-n for Most Significant Bits n (MSBs n) in the 8-bit vertical resolution with the range of [-128, 127], the number of sampling data of N = 10⁹, and the number of reference data of R = 10⁵. We use a notation of “MSBs n” for multiple bits generated from the 1st, 2nd, …, and nth MSB at one sampling point. The remaining bits from the (n+1)-th MSB, …, and 8th MSB are discarded from the original 8-bit data. The 8th MSB corresponds to the 1st least significant bit (LSB) of 8-bit data. For example, MSBs 3 corresponds to three bits, including the 1st, 2nd, and 3rd MSB of the 8-bit data. We also use a notation of “the nth MSB” for a single bit generated from only the nth MSB at one sampling point (see Section 4.2).

Figure 4 shows the comparison of the (ε,τ) entropy and the sample entropy for different MSBs n (i.e., different ε = 2^8-n ) when the vector length d is increased. τ is fixed at 100 ps. In Figure 4A, the (ε,τ) entropy decreases for a large d and large MSBs n (a small ε). Plateaus are not observed when d or n increases. By contrast, in Figure 4B, the sample entropy shows large plateau regions for all MSBs. The entropy at the plateau can be considered as the estimated value of the sample entropy. Therefore, the sample entropy can be estimated even for large MSBs n, and it is more reliable than the (ε,τ) entropy.

To investigate the difference between Figures 4A, B, we plot the probability of having the value of A i d = 0 in R reference data when d is increased for MSBs 4, as shown in Figure 5. In Figure 5A, the (ε,τ) entropy for MSBs 4 decreases as the probability of A i d = 0 increases for a large d, and the (ε,τ) entropy becomes zero when the probability of A i d = 0 reaches one. Therefore, it is difficult to correctly estimate the (ε,τ) entropy when the two neighboring vectors within the distance ε do not exist. By contrast, in Figure 5B, the sample entropy remains constant with an increase in d, even though the probability of A i d = 0 increases and is close to one. This result originates from the average calculation of A i d for R reference data before creating a logarithm, as shown in Eq. 5. Therefore, the sample entropy is more robust when the number of neighboring vectors within the distance ε is reduced for a large number of d and MSBs n.

From Figures 4, 5, we found that the sample entropy is less dependent on MSBs n than the (ε,τ) entropy. This result is explained by the comparison of Figures 5A, B, where the (ε,τ) entropy decreases as the probability of A i d = 0 increases, while the sample entropy remains the same value. Therefore, sample entropy can provide an entropy measurement for larger MSBs n. We consider this characteristic to be a general feature of other data used for entropy measurements.

Next, we change the quantization interval ε and the corresponding MSBs n. Figure 6 shows the comparison of the (ε,τ) entropy and the sample entropy obtained from the values of the plateaus shown in Figures 4A, B, where the vector length is set to d = 5. In Figure 6, the (ε,τ) entropy can be estimated reliably up to MSBs 3, because of the lack of the plateaus in Figure 4A. By contrast, the sample entropy can be obtained up to MSBs 8 (i.e., all 8 bits), as shown in Figure 4B. Figure 6 shows the maximum sample entropy (5.0) for MSBs 8. In addition, a sample entropy of more than one bit can be obtained for MSBs 4 or more MSBs. This result indicates that random number generation with uncertainty (i.e., entropy generation) can be achieved using one bit generated from MSBs 4 or larger.

We also change the sampling interval τ, corresponding to the inverse of the sampling frequency f _s (=1/τ). Figure 7 shows the sample entropy for different f _s values when n is changed. The quantization interval ε = 2^8-n is also changed. The vector length is set to d = 5. The sample entropy increases for large MSBs n. In addition, higher entropy is obtained when f _s is reduced. The sample entropy is saturated when f _s is reduced to 10 GS/s or smaller (τ = 100 ps or larger).

From Figure 7, the entropy rate is calculated by multiplying the estimated sample entropy by the sampling frequency. We can obtain an entropy up to MSBs 6 for f _s = 25 and 50 GS/s, and MSBs 8 for 10 GS/s or less, because of the existence of plateaus, as shown in Figure 4B. In Figure 7, a sample entropy of 5.0 at f _s = 10 GS/s is obtained for MSBs 8, and the corresponding entropy rate is 50 GHz (=5.0 × 10 GS/s). In another case, a sample entropy of 1.0 at 50 GS/s is obtained for MSBs 5, and the entropy rate is 50 GHz (=1.0 × 50 GS/s) as well. The same entropy rate can be obtained for different combinations of the sampling rate and number of MSBs. Therefore, an entropy rate of 50 GHz is obtained for the chaotic temporal waveforms of the laser intensity generated in the experiment.

To confirm the validity of the sample entropy estimation, we use the de facto standard of statistical tests of entropy evaluation for physical random number generators, known as NIST SP 800-90B (Barker and Kelsey, 2016). We obtain signals with n-bit vertical resolutions (MSBs n) for the NIST SP 800-90B tests and evaluate the minimum entropy of the n-bit signals generated from the physical entropy sources. The statistical tests consist of ten different entropy evaluations. The minimum value of the entropy evaluations in the ten tests is selected as the final estimate. We select the MSBs n of the 8-bit data (n bits) and evaluate the entropy of MSBs n. The maximum entropy corresponds to n for the data of MSBs n. We use 1 Mega point data to evaluate the entropy using NIST SP 800-90B.

Figure 8 compares the sample entropy and minimum entropy values obtained from NIST SP 800-90B. The sampling frequency is set to f _s = 10 GS/s (τ = 100 ps). Both of the entropy values increase as the number of MSBs increases. In addition, the sample entropy is smaller than that obtained using NIST SP 800-90B for all the MSBs. This result indicates that the sample entropy is underestimated compared with the entropy of NIST SP 800-90B. The sample entropy is close to the entropy of NIST SP 800-90B for MSBs 5 and 6. The overall characteristics coincides between the sample entropy and entropy of NIST SP 800-90B.

In this section, we evaluate the entropy rate of chaotic temporal waveforms used for physical random number generation. First, we evaluate the entropy of the noise signals to distinguish the origin of the entropy from stochastic noise and deterministic chaos. Figure 9A shows the temporal waveform of the experimentally obtained noise signals. The noise signal is detected using the digital oscilloscope and the photoreceiver without optical injection from the semiconductor laser (see Figure 2). The vertical range of the amplitude is set to be the same as that for the chaotic signals, as shown in Figure 3A.

Figure 9B shows the sample entropy of the noise signals obtained in the experiment. The sample entropy is zero between MSBs 1 and 5, and the entropy for more than MSBs 5 has a positive value. Here, MSBs 5 consists of the 1st, 2nd, …, and 5th MSB, and the 6th, 7th, and 8th MSB, which are the most sensitive to noise, are not included in MSBs 5. This result indicates that entropy originating from stochastic noise exists in the data of MSBs 6 or larger. Therefore, we use data of MSBs 5 or less for physical random number generation to avoid the contribution of stochastic noise to entropy.

We generate physical random bit sequences using minimum post-processing based on a bitwise XOR operation between the original and time-delayed signals (Takahashi et al., 2014; Sakuraba et al., 2015). First, chaotic temporal waveforms are sampled and converted into 8-bit signals. The 8-bit signal and its time-delayed signal are stored, and a bitwise XOR operation is performed between the original and time-delayed signals. The delay time is set to 3.0 ns (30 sampling points at the sampling interval of τ = 100 ps) to avoid the correlation between the original and time-delayed chaotic signals. The nth MSB of the resultant bits is extracted and combined with the signals sampled at different times as random bit sequences. Note that the nth MSB indicates only one bit at the nth bit counted from the 1st MSB, which differs from MSBs n used in the previous sections (i.e., MSBs n include the 1st, 2nd, …, and nth MSB). One bit obtained at the nth MSB is combined with different sampled data points, and N bits are generated from N sampled data points for each nth MSB.

To evaluate the statistical randomness of the generated bit sequences, we use NIST SP 800-22 tests (Rukhin et al., 2010), which are de facto standard tests for the statistical evaluation of random numbers. These statistical tests evaluate the statistical randomness of the bits (0 or 1) generated from the random number generators. NIST SP 800-22 consists of 15 statistical tests, and the random bits that pass all statistical tests are considered high-quality random numbers.

Figure 10A shows the results of NIST SP 800-22 at a sampling frequency of f _s = 50 GS/s. The bit sequences generated from the 5th MSB or higher can pass all 15 tests However, the bit sequences generated from the 4th MSB or less fails to some tests. This result indicates that the 5th MSB can be used as a random number. Figure 10B shows the results of NIST SP 800-22 for different sampling frequencies ranging from f _s = 2 to 25 GS/s. The 4th MSB or higher can pass all NIST tests in these cases. Therefore, the 4th MSB is also useful as a random number when f _s is reduced to 25 GS/s or less.