Multisensory 3D saliency for artficial attention systems

Lanillos, P., Ferreira, J.F. ORCID: 0000-0002-2510-2412 and Dias, J., 2015. Multisensory 3D saliency for artficial attention systems. In: 3rd Workshop on Recognition and Action for Scene Understanding (REACTS), Valletta, Malta, 5 September 2015.

Preview

Text 10576_Ferreira.pdf - Published version Download (2MB) | Preview

Abstract

In this paper we present proof-of-concept for a novel solution consisting of a short-term 3D memory for artificial attention systems, loosely inspired in perceptual processes believed to be implemented in the human brain. Our solution supports the implementation of multisensory perception and stimulus-driven processes of attention. For this purpose, it provides (1) knowledge persistence with temporal coherence tackling potential salient regions outside the field of view, via a panoramic, log-spherical inference grid; (2) prediction, by using estimates of local 3D velocity to anticipate the effect of scene dynamics; (3) spatial correspondence between volumetric cells potentially occupied by proto-objects and their corresponding multisensory saliency scores. Visual and auditory signals are processed to extract features that are then filtered by a proto-object segmentation module that employs colour and depth as discriminatory traits. We consider as features, apart from the commonly used colour and intensity contrast, colour bias, the presence of faces, scene dynamics and also loud auditory sources. Combining conspicuity maps derived from these features we obtain a 2D saliency map, which is then processed using the probability of occupancy in the scene to construct the final 3D saliency map as an additional layer of the Bayesian Volumetric Map (BVM) inference grid.

Item Type:	Conference contribution
Description:	Satellite event of the 16th International Conference of Computer Analysis of Images and Patterns (CAIP 2015).
Creators:	Lanillos, P., Ferreira, J.F. and Dias, J.
Date:	2015
Divisions:	Schools > School of Science and Technology
Record created by:	Jonathan Gallacher
Date Added:	04 Apr 2018 14:13
Last Modified:	04 Apr 2018 14:13
URI:	https://irep.ntu.ac.uk/id/eprint/33183

Actions (login required)

Edit View

Views

Downloads