In this paper, a new method to robustly and efficiently analyse video sequences to both extract foreground objects and to classify the static foreground regions as abandoned or removed objects (ghosts) is presented. As a first step, the moving regions in the scene are detected by subtracting to the current frame a referring model continuously adapted. Then, a shadow removing algorithm is used to find out the real shape of the detected objects and an homographic transformations is used to localize them in the scene avoiding perspective distortions. Finally, moving objects are classified as abandoned or removed by analysing the boundaries of static foreground regions. The method was successfully tested on real image sequences and it run about 7 fps at size 480x640 on a 2, 33 GB Pentium IV machine.