In this study, participants were tasked with a simple letter recognition exercise. They performed the task once on their own and once supposedly aided by an AI system. Half of the participants were told the system was reliable and it would enhance their performance, and the other half was told that it was unreliable and would worsen their performance.

‘In fact, neither AI system ever existed. Participants were led to believe an AI system was assisting them, when in reality, what the sham-AI was doing was completely random,’ explains doctoral researcher Agnes Kloft.

The participants had to pair letters that popped up on screen at varying speeds. Surprisingly, both groups performed the exercise more efficiently — more quickly and attentively — when they believed an AI was involved.

‘What we discovered is that people have extremely high expectations of these systems, and we can’t make them AI doomers simply by telling them a program doesn’t work,’ says Assistant Professor Robin Welsch.

Following the initial experiments, the researchers conducted an online replication study that produced similar results. They also introduced a qualitative component, inviting participants to describe their expectations of performing with an AI. Most had a positive outlook toward AI and, surprisingly even skeptical people still had positive expectations about its performance.

The findings pose a problem for the methods generally used to evaluate emerging AI systems. ‘This is the big realization coming from our study — that it’s hard to evaluate programmes that promise to help you because of this placebo effect’, Welsch says.

While powerful technologies like large language models undoubtedly streamline certain tasks, subtle differences between versions may be amplified or masked by the placebo effect — and this is effectively harnessed through marketing.

The results also pose a significant challenge for research on human-computer interaction, since expectations would influence the outcome unless placebo control studies were used.

‘These results suggest that many studies in the field may have been skewed in favour of AI systems,’ concludes Welsch.

The researchers will present the study at the CHI-conference on May 14.

Cat Royale is a unique collaboration between Computer Scientists from the University of Nottingham and artists at Blast Theory who worked together to create a multispecies world centred around a be-spoke enclosure in which three cats and a robot arm coexist for six hours a day during a twelve-day installation as part of an artist-led project. The installation was launched in 2023 at the World Science Festival in Brisbane, Australia and has been touring since, it has just won a Webby award for its creative experience.

The research paper, “Designing Multispecies Worlds for Robots, Cats, and Humans” has just been presented at the annual Computer-Human Conference (CHI’24) where it won best paper. It outlines how designing the technology and its interactions is not sufficient, but that it is equally important to consider the design of the `world’ in which the technology operates. The research also highlights the necessity of human involvement in areas such as breakdown recovery, animal welfare, and their role as audience.

Cat Royale centred around a robot arm offering activities to make the cats happier, these included dragging a ‘mouse’ toy along the floor, raising a feather ‘bird’ into the air, and even offering them treats to eat. The team then trained an AI to learn what games the cats liked best so that it could personalise their experiences.

“At first glance, the project is about designing a robot to enrich the lives of a family of cats by playing with them. ” commented Professor Steve Benford from the University of Nottingham who led the research, “Under the surface, however, it explores the question of what it takes to trust a robot to look after our loved ones and potentially ourselves.”

Working with Blast Theory to develop and then study Cat Royale, the research team gained important insights into the design of robots and its interactions with the cats. They had to design the robot to pick up toys, deploy them in ways that excited the cats, while it learned which games each cat liked. They also designed the entire world in which the cats and the robot lived, providing safe spaces for the cats to observe the robot and from which to sneak up on it, and decorating it so that the robot had the best chance of spotting the approaching cats.

The implication is designing robots involves interior design as well as engineering and AI. If you want to introduce robots into your home to look after your loved ones, then you will likely need to redesign your home.

Research workshops for Cat Royale were held at the Univeraity of Nottingham’s unique Cobotmaker Space where stakeholders were bought together to think about the design of the robot /welfare of cats. Eike Schneiders, Transitional Assistant Professor in the Mixed Reality Lab at the University of Nottingham worked on the design, he said: “As we learned through Cat Royale, creating a multispecies system — where cats, robots, and humans are all accounted for — takes more than just designing the robot. We had to ensure animal wellbeing at all times, while simultaneously ensuring that the interactive installation engaged the (human) audiences around the world. This involved consideration of many elements, including the design of the enclosure, the robot and its underlying systems, the various roles of the humans-in-the-loop, and, of course, the selection of the cats.”