One of the major criteria for the scientificity of a research study is reproducibility. In this paper, we will present the main definitions around reproducibility. We will examine to what extent computer science and simulation works are concerned. Anyone wishing to produce quality scientific work will pay attention to the numerical reproducibility of his simulation results. Significant differences can be observed in the results if the practitioner does not apply best practices. In addition to reproducibility, we first need repeatability to debug our simulation software. We will see that even for parallel stochastic simulations, it is possible to reproduce the same numerical results by implementing a rigorous method tested with a billion threads. It is possible to check parallel results with their sequential counterpart before scaling, thus gaining confidence in the proposed simulation results.