Recent studies have highlighted that people with mathematical and scientific talents develop a different brain structure from those with typical development. However, most of these studies have focused on the relationship between cognitive functions of the brain and the operation of a single area of the brain. This study explores the connections among the network structures with relation to mathematical and scientific talents, intelligence, and white matter. The study recruited 42 men with normal nerve functions. The experimental group comprised 21 participants with mathematical and scientific talents and an age of 21.00 ± 1.67 years; the control group comprised 21 participants with typical developmental and an age of 21.48 ± 2.29 years. The mathematical and scientific talent and typical developmental groups consisted of 21 people each. The researchers adopted the third version of the Wechsler Adult Intelligence Test to evaluate individual intelligence, conduct diffusion tensor imaging of participants, and construct a network of white matter to analyze the overall network attributes and nodal efficiencies using graph theory. The results show that the communication efficiency among the nodes inside the local region is relatively better in people with mathematical and scientific talents, particularly the in the superior prefrontal gyrus. Moreover, when intelligence was equal between the two groups, the mathematical and scientific talent group outperformed the other in terms of node efficiency in local regions and the clustering coefficient in the superior occipital gyrus. The relationship between the topological properties and the intelligence of the mathematical and scientific talent group and the typical developmental group showed that only the intelligence of the typical developmental group was positively connected with integrated efficiency across several regions of the brain; however, no direct correlation was shown in the mathematical and scientific talents group. The results not only provided empirical evidence for the disparity in white matter structure between mathematical and scientific talent and typical development groups but also distinguished mathematical and scientific talents, mathematical ability, and intelligence based on the topological network of the brain, which can be used in future assessments for people with mathematical and scientific talents.