Autonomous railway-based systems have been existing already for some time, especially in closed environments, which have been controlled by external infrastructure-based solutions. These include for example metros and airport shuttles. The first real world implementations of autonomous trains are reaching large-scale piloting stages and initial results have been promising. If one wants to increase the level of autonomy in railway operations, like trains and trams, and operate those in a complex dynamic urban environment, the requirements are much tighter. When you add reckless humans, who are crossing tracks randomly wherever they want, to the equation, autonomous or even semi-autonomous railway operations, especially in the urban areas, require highly sophisticated sensory systems for continuous environmental monitoring and situational awareness. Usually, at least cameras, LiDAR and radar are included in that set for real-time assessment of the traffic situation to provide the machine enough time to react in all weather conditions. We will provide your machine the much-needed Situational awareness (GIM-Observer).If we want to increase the level of autonomy to a level where we can consider the removal of the human driver from the machine, we must give the machine the capability to localize itself autonomously everywhere on the route map. Due to the urban environments, including tunnels and proximity of tall buildings, there will always be areas where the use of traditional GNSS based localization is just not possible due to the lack of reliable satellite connection. To secure the full localization coverage, an additional system must be included. This localization system is usually based on a detailed constantly updated feature-based map of the working environment, sensory system capable of detecting those selected features in all conditions, and the algorithms to produce probabilistic real-time 6D position estimates. 3D localization (GIM-Locator) and environmental modeling (GIM-Mapper) sensor and software systems provide the foundations for situational awareness, and therefore obstacle detection and the following collision warning signals for the driver.