The stereo line-scan camera system has various arrangement modes [7]. For our system, the side-by-side mode is chosen. As depicted in Figure 1A, the system mainly includes two line-scan cameras (Dalsa, spL2048-140km) mounted lens (Nikon, AF Nikkor 14 mm f/2.8D), two line lasers, a industrial computer with two frame grabbers (Dalsa, OR-X4C0-XPF00), several mechanical and calibration devices. Like most stereo line-scan camera systems [2, 4–7, 29], our system also requires the two line-scan cameras to be coplanar aligned manually using a suitable mechanical device so that the two viewing planes determined by the projection center and the pixel elements are approximately coplanar. Consequently, the measurement field is the overlapping area of the two planes.

It is necessary for an imaging model to define the coordinate systems in order to describe the intrinsic and extrinsic parameters. As shown in Figure 1B, the world coordinate system, the two camera coordinate systems and the two pixel coordinate systems are denoted by O w − X w Y w Z w , O c 1 − X c 1 Y c 1 Z c 1 , O c 2 − X c 2 Y c 2 Z c 2 , o 1 − u 1 , o 2 − u 2 , respectively. [ R , T ] are extrinsic parameters between the camera 1 and 2 coordinate systems, and [ R 1 , T 1 ] and [ R 2 , T 2 ] are extrinsic parameters between the world coordinate system and the camera coordinate system, where R , R 1 , R 2 and T , T 1 , T 2 are the rotation and translation matrixes, respectively.

For the line-scan camera, perspective projection is only satisfied along the pixel elements direction, i.e., u 1 and u 2 . According to perspective projection [2, 11], the relationship between a point P c k ( k = 1,2 ) on the viewing plane and its image points p k ( u k ) can be described as Z c k [ u k 1 ] = [ α k u k 0 0 0 1 0 ] [ X c k Z c k 1 ] (1) where α k and u k 0 are the intrinsic parameters and denote the focal length in pixel units and the coordinate of the principle point of the camera k, respectively; ( X c k , Z c k ) are the coordinates of P c k reference to the camera k coordinate systems, respectively.

Because a world point P w only appears on the viewing planes can be imaged by the stereo line-scan cameras, for convenience, the O w − X w Y w is required to coincide with the viewing planes. Therefore, ( X w , Y w ) denote the 2D coordinates in the measurement field. Furthermore, the world coordinate system and the camera k coordinate system can be related in the following ways [ X c k Y c k Z c k 1 ] = [ R k T k 0 1 ] [ X w Y w Z w 1 ] = [ − sin ⁡ θ k cos ⁡ θ k 0 t k x 0 0 ( − 1 ) k + 1 0 ( − 1 ) k + 1 ⁡ cos ⁡ θ k ( − 1 ) k + 1 ⁡ sin ⁡ θ k 0 t k z 0 0 0 1 ] [ X w Y w 0 1 ] ⇒ [ X c k Z c k 1 ] = [ − sin ⁡ θ k cos ⁡ θ k t k x ( − 1 ) k + 1 ⁡ cos ⁡ θ k ( − 1 ) k + 1 ⁡ sin ⁡ θ k t k z 0 0 1 ] [ X w Y w 1 ] (2) where θ k is a rotation angle and [ t k x     t k z ] T is a translation vector.

Taking Eq. 2 into Eq. 1, we can get { Z c 1 [ u 1 1 ] = [ − α 1 ⁡ sin ⁡ θ 1 + u 10 ⁡ cos ⁡ θ 1 α 1 ⁡ cos ⁡ θ 1 + u 10 ⁡ sin ⁡ θ 1 α 1 t 1 x + u 10 t 1 z cos ⁡ θ 1 sin ⁡ θ 1 t 1 z ] [ X w Y w 1 ] = [ h ′ 11 h ′ 12 h ′ 13 h ′ 21 h ′ 22 h ′ 23 ] [ X w Y w 1 ] , Z c 2 [ u 2 1 ] = [ − α 2 ⁡ sin ⁡ θ 2 − u 20 ⁡ cos ⁡ θ 2 α 2 ⁡ cos ⁡ θ 2 − u 20 ⁡ sin ⁡ θ 2 α 2 t 2 x + u 20 t 2 z − cos ⁡ θ 2 − sin ⁡ θ 2 t 2 z ] [ X w Y w 1 ] = [ h ″ 11 h ″ 12 h ″ 13 h ″ 21 h ″ 22 h ″ 23 ] [ X w Y w 1 ] (3)

By substituting the second equation into the first equation of (3) and substituting the fourth equation into the third equation of (3), Eq. 3 can be transformed into the following descriptions { X w h ′ 11 + Y w h ′ 12 + h ′ 13 − u 1 X w h ′ 21 − u 1 Y w h ′ 22 − u 1 h ′ 23 = 0 , X w h ″ 11 + Y w h ″ 12 + h ″ 13 − u 2 X w h ″ 21 − u 2 Y w h ″ 22 − u 2 h ″ 23 = 0 (4)

Obviously, the imaging model (3) or (4) can depict the relationship between the 2D world coordinate ( X w , Y w ) and its corresponding 1D pixel coordinates ( u 1 and u 2 ) of the image pairs.

It is easy to see that the imaging model (3) or (4) is linear. However, the actual imaging is a nonlinear process inevitably affected by different kinds of lens distortions [2]. To improve the model performance and measurement accuracy, lens distortion is taken into consideration.

For the area-scan cameras, the mathematical expressions for three types of lens distortions are as follows [9, 31]. { x ˜ ′ = x ˜ + x ˜ ( k 1 r ˜ 2 + k 2 r ˜ 4 + k 3 r ˜ 6 ) + 2 p 1 x ˜ y ˜ + p 2 ( 3 x ˜ 2 + y ˜ 2 ) + s 1 ( x ˜ 2 + y ˜ 2 ) , y ˜ ′ = y ˜ + y ˜ ( k 1 r ˜ 2 + k 2 r ˜ 4 + k 3 r ˜ 6 ) + p 1 ( x ˜ 2 + 3 y ˜ 2 ) + 2 p 2 x ˜ y ˜ + s 2 ( x ˜ 2 + y ˜ 2 ) (5) where r ˜ = x ˜ 2 + y ˜ 2 ; ( x ˜ , y ˜ ) = ( X c / Z c , Y c / Z c ) is a distortion-free normalized image coordinates; ( x ˜ ′ , y ˜ ′ ) is the corresponding distorted normalized image coordinates; ( k 1 , k 2 , k 3 ) , ( p 1 , p 2 ) , ( s 1 , s 2 ) are the coefficients of radial, decentering and thin prism distortions, respectively.

Considering the specialty of line-scan camera [32], y ˜ ≈ 0 and thus lens distortion model (5) can be simplified as x ˜ ′ = x ˜ + ( s 1 + 3 p 2 ) x ˜ 2 + k 1 x ˜ 3 + k 2 x ˜ 5 + k 3 x ˜ 7 (6)

If we ignore the third radial distortion and set k 0 = s 1 + 3 p 2 , Eq. 6 can be furthermore simplified to x ˜ ′ k = x ˜ k + k k 0 x ˜ k 2 + k k 1 x ˜ k 3 + k k 2 x ˜ k 5 (7)

Considering the lens distortion, Eq. 1 can be rewritten as [ u k 1 ] = [ α k u k 0 0 1 ] [ X c k Z c k 1 ] = [ α k u k 0 0 1 ] [ x ˜ k 1 ] ⇒ [ u ′ k 1 ] = [ α k u k 0 0 1 ] [ x ˜ ′ k 1 ] (8)

Substituting Eq. 8 into Eq. 4, we can derive the real distorted imaging model { X w h ′ 11 + Y w h ′ 12 + h ′ 13 − ( u 1 ′ − α 1 k 10 x ˜ 1 2 − α 1 k 11 x ˜ 1 3 − α 1 k 12 x ˜ 1 5 ) ( X w h ′ 21 + Y w h ′ 22 + h ′ 23 ) = 0 , X w h ″ 11 + Y w h ″ 12 + h ″ 13 − ( u 2 ′ − α 2 k 20 x ˜ 2 2 − α 2 k 21 x ˜ 2 3 − α 2 k 22 x ˜ 2 5 ) ( X w h ″ 21 + Y w h ″ 22 + h ″ 23 ) = 0 (9)

As shown in Figure 2A, our designed calibration target comprises a group of magnetic rods (8 collinear feature points with 100 mm spacing) perpendicular to a flat iron plate. The plate is adjusted to be parallel to the viewing plane and pasted by a coordinate map which can give the 2D world coordinates ( X w , Y w ) when the target translates. It can be seen from Figure 2A that the calibration target has a simple structure and auxiliary devices, such as linear stage and area-scan camera, do not need to assist the calibration. Therefore our target is low cost.

A two-step algorithm is proposed to determine the optimal values of intrinsic, extrinsic and distortion parameters. Firstly, we will compute the initial values of intrinsic and extrinsic camera parameters by solving the imaging model (4) with given feature points. Secondly, the initial values will be refined by a non-linear optimization technique to greatly improve the calibration accuracy.

Given n ( n ≥ 2 ) translations of the calibration target, the coefficients of H 1 = [ h ′ 11 , h ′ 12 , h ′ 13 , h ′ 21 , h ′ 22 , h ′ 23 ] T and H 2 = [ h ″ 11 , h ″ 12 , h ″ 13 , h ″ 21 , h ″ 22 , h ″ 23 ] T of Eq. 4 can be solved by the system of homogeneous equations W k H k = 0 ( 8 n ) (10) with W k = [ X w ( 1 ) Y w ( 1 ) 1 − u k ( 1 ) X w ( 1 ) − u k ( 1 ) Y w ( 1 ) − u k ( 1 ) ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ X w ( 8 ) Y w ( 8 ) 1 − u k ( 8 ) X w ( 8 ) − u k ( 8 ) Y w ( 8 ) − u k ( 8 ) ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ X w ( 8 n ) Y w ( 8 n ) 1 − u k ( 8 n ) X w ( 8 n ) − u k ( 8 n ) Y w ( 8 n ) − u k ( 8 n ) ] (11) where u k ( 1 ) , ⋯ , u k ( 8 ) , ⋯ , u k ( 8 n ) can be obtained by a sub-pixel location algorithm that can detect the centers of 8 rods in n captured images.

The matrix W k can be decomposed by using singular value decomposition (SVD). The solution to Eq. 10 is the eigenvector of W k T W k with the smallest eigenvalue. Since H k is only defined up to a scale factor, we can get a solution H ˜ k = H k / H k e n d = [ h ˜ k 1 , h ˜ k 2 , h ˜ k 3 , h ˜ k 4 , h ˜ k 5 , 1 ] T , where H k e n d is the end element of H k . The following expressions can be given by recalling that Eq. 3 t k z [ h ˜ k 1 h ˜ k 2 h ˜ k 3 h ˜ k 4 h ˜ k 5 1 ] = [ − α k ⁡ sin ⁡ θ k + ( − 1 ) k + 1 u k 0 ⁡ cos ⁡ θ k α k ⁡ cos ⁡ θ k + ( − 1 ) k + 1 u k 0 ⁡ sin ⁡ θ k α k t k x + u k 0 t k z ( − 1 ) k + 1 ⁡ cos ⁡ θ k ( − 1 ) k + 1 ⁡ sin ⁡ θ k t k z ] (12)

Solving this equation, the intrinsic and extrinsic parameters can be obtained { t k z 2 h ˜ k 4 2 + t k z 2 h ˜ k 5 2 = 1 ⇒ t k z = ± 1 / h ˜ k 4 2 + h ˜ k 5 2 , α k = ( − 1 ) k t k z 2 ( h ˜ k 1 h ˜ k 5 − h ˜ k 2 h ˜ k 4 ) , u k 0 = t k z 2 ( h ˜ k 1 h ˜ k 4 + h ˜ k 2 h ˜ k 5 ) , θ k = tan − 1 ( h ˜ k 5 / h ˜ k 4 ) , t k z h ˜ k 3 = α k t k x + u k 0 t k z ⇒ t k x = t k z ( h ˜ k 3 − u k 0 ) / α k , R k = [ ( − 1 ) k t k z h ˜ k 5 ( − 1 ) k + 1 t k z h ˜ k 4 0 0 0 ( − 1 ) k + 1 t k z h ˜ k 4 t k z h ˜ k 5 0 ] , T k = [ t k x 0 t k z ] (13)

Obviously, according to the coordinate systems defined in Figure 1B, we can get t k z > 0 .

In the initial calculation of the parameters, the distortion is neglected. In order to obtain more accurate calibration parameters, a non-linear optimization technique such as Levenberg-Marquardt algorithm is employed to refine the camera parameters. The goal is to find out the optimal intrinsic, extrinsic and distortion parameters which minimize the sum of squares of reprojection errors, i.e., min α k , u k 0 , R k , T k , k k 0 , k k 1 , k k 2 ∑ j = 1 8 n ‖ u ′ k ( j ) − u ′ ^ k ( j ) ( X w ( j ) , Y w ( j ) ) ‖ 2 (14) where u ′ ^ k j ( X w j , Y w j ) is a function which projects the world points ( X w j , Y w j ) into the image points according to the real distorted imaging model (9). For the initial values of distortion parameters, we may take k k 0 = k k 1 = k k 2 = 0 since the lens distortion is usually small and close to zero.

Once the parameters of two line-scan cameras are respectively refined, we can furthermore minimize the sum of squares of reprojection errors for each camera, i.e., min α k , u k 0 , R k , T k , R , T , k k 0 , k k 1 , k k 2 ∑ k = 1 2 ∑ j = 1 8 n ‖ u ′ k ( j ) − u ′ ^ k ( j ) ( X w ( j ) , Y w ( j ) ) ‖ 2 (15) to obtain the final accurate parameters of stereo line-scan cameras, where the initial values of R , T can be calculated by R = R 2 R 1 − 1 = R 2 R 1 T , T = T 2 − R T 1 .

To evaluate the effectiveness of the proposed method, we calibrate our stereo line-scan camera system. It can be observed from Figures 1A and 2A, when the calibration target is translated on the flat plate, the coordinate map can give the 2D world coordinates ( X w , Y w ) for camera calibration. In our experiment, n = 7 , that is to say we translate target seven times, a total of 56 ( 8 × 7 ) feature points. Figure 2B exhibits two of seven image pairs of 8 magnetic rods illuminated by the line lasers and the sub-pixel locations of rod centers. The first and third images are captured by camera 1 while the second and fourth images are captured by camera 2. We set the line-scan cameras to 300 rows per frame, so the image resolution is 2048 × 300 .

Table 1 shows the calibration results of stereo line-scan cameras. We implement our method in Matlab. The root mean square (RMS) of reprojection errors of two camera is less than 0.40 pixel. It is well mentioning that the time of our calibration process is no more than 5 min for both cameras while the time of the method [4] for each camera is about 30 min. The time required for our calibration process is greatly reduced mainly due to the help of the simple target and simple translation operation. Therefore it can be said that our proposed method is significantly effective.

Furthermore, our calibration parameters are verified by measuring the 2D coordinates in the measurement field with a range of 1000   mm × 800   mm .

The 2D coordinates can be triangulated by solving Eq. 4, then [ X w Y w ] = [ u 2 h ″ 22 − h ″ 12 − u 1 h ′ 22 + h ′ 12 − u 2 h ″ 21 + h ″ 11 u 1 h ′ 21 − h ′ 11 ] × [ h ′ 13 − u 1 h ′ 23 h ″ 13 − u 2 h ″ 23 ] ( u 1 h ′ 21 − h ′ 11 ) ( u 2 h ″ 22 − h ″ 12 ) − ( u 1 h ′ 22 − h ′ 12 ) ( u 2 h ″ 21 − h ″ 11 ) (16) where u k is distortion-free pixel coordinates which can be related to x ˜ k by Eq. 8. It is noticed that x ˜ k should be solved through Eq. 7 using an iterative method since x ˜ ′ k can be obtained.

A comparison between our and [4]’s measurement results is shown in Table 2. We test a total of 21 2D points similar to [4]. According to the results, it can be seen obviously that the maximum errors of our method for X w and Y w are 0.84 and 0.67 mm respectively, while the maximum errors of [4]’s method for X w and Y w are 1.41 and 1.07 mm, respectively. The RMS errors of X w and Y w of our method are 0.449 and 0.373 mm respectively, which are also less than [4]’s method. Generally, these errors can be more or less affected by the accuracy of coordinate map, the planeness of iron plate, the precision of calibration target, the parallelism of two viewing planes and iron plate, and the sub-pixel location algorithm.

In conclusion, the experimental results demonstrate that our proposed method can provide satisfactory 2D coordinate measurement and has much more superiority both in the calibration time and accuracy compared with the [4]’s method.