Most existing sandstorm image enhancement methods are based on traditional theory and prior knowledge, which often limit their generalization ability in real-world applications. In addition, these approaches often adopt a strategy of color correction followed by dust removal, which makes the algorithm structure too complex. To solve the issue, we propose a novel image restoration model, named all-in-one sandstorm removal network (AOSR-Net). It is designed based on a re-formulated sandstorm scattering model, which directly establishes the image mapping relationship by integrating intermediate parameters. Such integration strategy can effectively solve the issues of over-enhancement and weak generalization in the field of sand dust image enhancement. Experimental results on both synthetic and real-world sandstorm images illustrate that the proposed AOSR-Net is superior than the state-of-the-art methods.