Detection of crisis in socio-material systems
via VISual-COgnitive-SOcial processes

36 months (extended of 9 months)

The project is financed by the Provincia Autonoma di Trento through the scheme Research Unit 2011 for a total amount of 600K euros.

Research Team

Principal InvestigatorRoberta Ferrario
Post-Docs Chiara Bassetti
Cinzia Giorgetta
Daniele Porello
Francesco Setti
Nicola Zeni
PhDs Davide Conigliaro
Emilio Sanfilippo
Visitors (2013) Błażej Skrzypulec
External Collaborators Marco Cristani

Description

The project will concentrate on the detection of critical situations that happens in the entanglement of visual-cognitive-social processes, the process of seeing a scene, forming a belief or an expectation and engaging in an interaction. We argue that, far from being a linear process, this is rather an entanglement, in which vision influences what agents believe, their beliefs determine them to interact in a certain way and sustain their institutions, the rules they are subject to or their previous interactions may let emerge beliefs and expectations, that may influence what they see and how they see it, and so on. Moreover, in a socio-material system, both humans and technical devices participate in and enact such entanglement. Many disciplines, ranging from philosophy to cognitive sciences, artificial intelligence and sociology, have dealt with such phenomena, often focusing only on a part of the entanglement and neglecting the others.

The scenario of the project is that of designed socio-material systems, intended as complex systems embedding human and artificial agents, material and institutional artifacts and resources, in which behavior and practices are partly constrained by norms. The complexity of such systems makes it very hard for them to cope with critical situations, as they emerge from the interplay between all participants and cannot be reduced to mere technical malfunctioning or to the negligence/malevolence of the human actors involved, but should be rather dealt with at a systemic level.

The challenge addressed by this project is that of enabling a wider and more comprehensive perspective by leveraging on the results of the involved disciplines, thus allowing for the implementation of socio-material systems that are aware of the criticalities that can emerge in the entanglement of visual- cognitive-social processes. An ontological model will be built that captures the insights coming from different disciplines, reached with their own specific paradigms, making use of their own concepts and methodologies. The conceptual clarification resulting from the ontological analysis will enable:

  • scholars of the various disciplines involved to better understand each other, thus enhancing knowledge on their specific field by cross-fertilization of studies
  • designers of socio-material systems to be more aware of the entanglement, thus preventing as much as possible critical situations and preparing a systemic reaction to them
  • agents (human and artificial) that participate in the system to better prevent, recognize, recover and learn from critical situations, and to infer new unexpected critical situations “at run time”, while the system evolves, by making it in a sense self-transparent.

The expected outcome will be a theoretical artifact, but meant to be practically used within the system and co-evolving with it, and enabling the agents in the system to collectively manage critical situations thanks to an enhanced self- transparency.

For the part concerning the integration of the ontological approach and computer vision, VisCosSo is being carried out in collaboration with the VIPS Laboratory (Vision, Image Processing & Sound) of the Dipartimento di Informatica at the University of Verona, with the supervision of Marco Cristani.

Results

The main research results of VisCoSo are contained in our Publications (see below). Moreover, we have built two datasets, S-Hock and RepTile, which are publicly available for the scientific community. This article also summarizes the main achievements of the project.

Publications

Articles on the Project in general

Articles on Ethnography, Interaction and Case Study

Articles on Psychology and Decision Making

  • Giorgetta, C., Grecucci, A., Ferrario, R., Bonini, N., Sanfey, A. G., Emotion Regulation Strategies on Risky Decision Making. Rome Workshop on Experimental Psychopathology, March 20 – 21, 2015.

Articles on Computer Vision and Pattern Recognition

Articles on Philosophical Foundations

Articles on Logics and Knowledge Representation

Articles on Ontology

  • Ferrario, R., Porello, D., Towards a conceptualization of socio-material entanglement, in Christiansen, H., Stojanovic, I., Papadopoulos, G. (eds.), Modeling and Using Context. The Ninth International and Interdisciplinary Conference on Modeling and Using Context, CONTEXT 2015, Volume 9405 Lecture Notes in Computer Science, 2015, 32-46.
  • Porello, D., Bottazzi, E., Ferrario, R., Group Conflict as Social Contradiction, in D’Errico, F., Poggi, I., Vinciarelli, A., Vincze, L. (eds.), Conflict and Multimodal Communication, Volume 1 Social Research and Machine Intelligence, 2015, 33-53.
  • Porello, D., Cristani, M., Ferrario, R., Integrating Ontologies and Computer Vision for Classification of Objects in Images, in Besold, T. R.., Kühnberger, K.-U. (eds.) Proceedings of the Workshop on Neural-Cognitive Integration. KI 2015 38th German Conference on Artificial Intelligence, Publication Series of the Institute of Cognitive Science PICS, University of Osnabrück, Vol. 1-2015, 1-15.
  • Porello, D., Bottazzi, E., Ferrario, R., The Ontology of Group Agency, in Garbacz, P., Kutz, O. (eds.), Formal Ontology in Information Systems, 8th International Conference on Formal Ontology in Information Systems (FOIS 2014), Volume 267 Frontiers in Artificial Intelligence and Applications, IOS Press, 2014,183-196.
  • Porello, D., Endriss, U., Ontology Merging as Social Choice: Judgment Aggregation under the Open World Assumption, Journal of Logic and Computation, 24 (6): 1229-1249, 2014.
  • Porello, D., Setti, F., Ferrario, R., Cristani, M., Multiagent socio-technical systems: An ontological approach, in T. Balke (eds.), Coordination, Organisations, Institutions and Norms in Agent Systems IX (COIN 2013), Volume 8386 Lecture Notes in Artificial Intelligence, Springer 2013, 42-62.
  • Porello, D., Ferrario, R., Giorgetta, C., Ontological modeling of emotion-based decisions, in K.U. Kühnberger, P. König, S. Walter (eds.), Proceedings of the workshop Formalizing Mechanisms for Artificial General Intelligence and Cognition (FormalMAGIC-2013), Publication Series of the Institute of Cognitive Science PICS, University of Osnabrück, Vol. 1-2013, 17-23.