This paper proposes a network model based on a three-stream network and improved attention mechanism for blind image quality assessment (TSAIQA). The inputs of the three streams are the distorted image, the pseudoreference image obtained by the improved generative adversarial network (GAN), and the gradient map of the distorted image. The distorted image stream focuses on the holistic quality-related features, the pseudoreference image stream is used to supplement the lost features due to distortion, and the gradient stream explicitly extracts the quality-related structural features. In addition, spatial and channel attention mechanisms combining first- and second-order information are proposed, and the improved attention mechanisms are applied to the three-stream network to optimize spatial and channel-level features effectively. Finally, the three-stream fusion features are input to the quality regression network to predict the image quality. To demonstrate the effectiveness of the proposed model, experiments are conducted on four classical IQA databases and two new large-scale databases. The experimental results show that the results of our TSAIQA model outperform the most advanced IQA methods and confirm the effectiveness of the proposed network structure and attention mechanisms.