In this paper we propose a method for accurate ego-lane localization using camera images, on-board sensors and lanes number information from OpenStreetMap (OSM). The novelty relies in the probabilistic framework developed, as we introduce a modular Bayesian Network (BN) to infer the ego-lane position from multiple inaccurate information sources. The flexibility of the BN is proven, by first, using only information from surrounding lane-marking detections and second, by adding adjacent vehicles detection information. Afterward, we design a Hidden Markov Model (HMM) to temporary filter the outcome of the BN using the lane change information. The effectiveness of the algorithm is first verified on recorded images of national highway in the region of Clermont-Ferrand. Then, the performances are validated on more challenging scenarios and compared to an existing method, whose authors made their datasets public. Consequently, the results achieved highlight the modularity of the BN. In addition, our proposed algorithm outperforms the existing method, since it provides more accurate ego-lane localization: 85.35% compared to 77%.