We proposed a complex defect inspection (CDI) technique for quality control of transparent substrates that uses the diffraction characteristics of digital holograms and a machine learning algorithm. A complex pattern diffraction model was built to provide two diffraction criteria, the least separation of confusion and the effective diffraction distance, to extend depth of focus in the effective diffraction regime for numerical reconstruction. On the basis of an analysis of three-dimensional diffraction characteristics of complex images, defect identification was performed to detect and classify defects (cracks, dusts, and watermarks) in transparent substrates using region-based segmentation and a machine learning algorithm. The experimental results indicated that the defect detection performance of the proposed CDI system was recall = 96.3% and precision = 92.8%. Moreover, overall multiclass classification accuracy = 95.3%, resulting in a discrimination area under the receiver operating characteristic curve (Az) of 0.96.
