Butterflies’ recognition serves a crucial role as an environmental indicator and a key factor in plant pollination. The automation of this recognition process, facilitated by Convolutional Neural Networks (CNNs), can expedite this task. Several pre-trained CNN models, such as VGG, ResNet, and Inception, have been widely used for this purpose. However, the scope of previous research has been somewhat constrained, focusing only on a maximum of 15 classes. This study proposes to modify the CNN InceptionV3 model and combine it with three data augmentations to recognize up to 100 butterfly species. To curb overfitting, this study employs a series of data augmentation techniques. In parallel, we refine the InceptionV3 model by reducing the number of layers and integrating four new layers. The test results demonstrate that our proposed model achieves an impressive accuracy of 99.43% for 15 classes with only 10 epochs, exceeding prior models by approximately 5%. When extended to 100 classes, the model maintains a high accuracy rate of 98.49% with 50 epochs. The proposed model surpasses the performance of standard pre-trained models, including VGG16, ResNet50, and InceptionV3, illustrating its potential for broader application.