In order to more accurately diagnose rice phosphorus nutrition and help rice growth, an improved ShuffleNet V2 rice phosphorus nutrition diagnosis method is proposed. This method improves the model by introducing the ECA attention mechanism into the ShuffleNet V2 network model, and selects the pooling method of Attention Pooling to optimize model training. The transfer learning strategy is adopted to transfer the pre-trained weights on the ImageNet large dataset to the improved ShuffleNet V2 network model to train the rice leaf dataset and construct a rice phosphorus nutrition diagnosis model. The experimental results showed that the improved ShuffleNet V2 network model had higher accuracy, precision, recall and F1 value than other comparative network structure models in the rice tillering stage and rice jointing stage, and the training parameters were small, the training was more stable, and the convergence speed was faster. It proved that the improved ShuffleNet V2 rice phosphorus nutrition diagnosis model had good diagnostic recognition ability, which was helpful to adopt scientific and effective fertilization strategies under big data. At the same time, the improved ShuffleNet V2 network model had also achieved remarkable results on the Plant Village public dataset, which further verified the effectiveness and generalization ability of the improved model.