Imagining a building with heart

While we have been self-isolating, the innovative Melbourne Connect precinct has been taking form. At its heart: a major new digital artwork that will live and grow with the building, its occupants, and its environment over time.

Lead artist Robert Walton (VCA, CIS), research data specialist Zaher Joukhadar (MDAP), and project manager Lindsay Bick (MC) explain the beating heart of Melbourne Connect (MC).

Lindsay, can you tell us about your brief to artists?

For this extraordinary building project, we were looking for an interactive digital art installation for the foyer of the main building but found ourselves short on necessary skills in the project team who are not artists.

“We wanted the feature art installation to be equally interesting whether it be your first time visiting, or your thousandth.”

We received a broad range of responses, with experiences that each had many elements, but Robert’s Heart concept delivered quite a few experiences in just one installation, and that impressed the judges,

Melbourne Connect is a ‘purpose-built innovation precinct. The people, place, and programs at Melbourne Connect are curated to unlock digitally driven, data-enabled, and socially responsible solutions to our most pressing future challenges’. Images: Melbourne Connect

Robert, what did you think of Lindsay’s brief?

Well, four of us — Zaher, Dr Eduardo Velloso and Henrietta Lyons from Interaction Design Lab and I  decided to quickly form a team to develop my initial idea ‘The Heart’. In 2020, only Zaher and I have the capacity to work on this project, but our complementary skill sets have allowed us to continue to develop and refine the concept.

Our team has now expanded to include some of Melbourne’s most talented independents including creative tech lead and co-founder of Flight Plan Gendry Morales, lighting designer, and co-founder of Additive Paul Lim, maker-maestro Justin Green, and creative programmer and co-founder of EXP Brad Hammond.

Why The Heart?

Well, it’s a good question. The artwork is a provocation to wonder:

“What does it mean for a building to have a heart? What is the risk of working with(out) heart? Where is The Heart of a building and its community?”

The Heart idea emerged from the brief to work with this extraordinary building filled to the brim with sensors. This made me realise that the building itself feels things. We often think of sensors equating to thinking, but prior to thought there is feeling, and becoming accustomed to sensations. Modern ‘smart’ buildings are life support systems on the verge of becoming beings in their own right at the dawn of the AI age: sensitive to activities within and around them.

So the artwork’s provocation is to reveal its heart, to show something of its workings, and to invite you to do the same. Melbourne Connect brims with extraordinary brains. My provocation is to attend to our hearts. The artwork seeks to nurture a sense of life together with the building.

“And that leads to all kinds of cool conversations, which we’ve had, about emerging artificial intelligence (AI), and how buildings of the future may be on the verge of a kind of consciousness.”

Maybe our future buildings will have a kind of benevolence. When we think of AI we often think of horror films and scary robots, but actually, lots of these kinds of systems are about supporting life and creating optimum experiences for us to live in work. They nurture us.

READ: The Digital in Architecture: Then, Now and in the Future. Image: Space10

The Heart will start beating when MC opens and will continue until the end of the lease in 42 years, or maybe forever. The Heart rate will go up and down throughout the day as people come and go, work and play, as the weather changes and the sun moves across its walls.

Maybe it flutters a little bit when it meets a new person. Maybe it remembers people and holds onto these little fleeting moments in our lives. But it is always there, remembering and feeling.

We tend to think of sensors as ways to make computers think but actually, the sensing part comes first.

Wow. I see why you won!

[Group giggles]

Zaher, let’s talk about your critical part in helping to realise this project.

Yeah! So, my role is to help Robert to realise the implementation of such an experimental project. I’m focussed on making sense of the data journey from sensor collection to visualisation.

“I have been analysing, processing, and aggregating data from multiple sensors to see what meaningful or beautiful visualisation we can create in the installation.”

Next, I will work with the lighting designer and Unity expert to help inform the visualisation of the live building data.

 

READ: Introducing Zaher, one of our talented research data specialists with a passion for developing boundary-pushing projects at the intersection of art, science, and technology. Image: Zaher Joukhadar

Alright. Let’s learn more about The Heart. Which data will The Heart respond to, Lindsay?

The Heart will respond to many different types of data including temperature, humidity, carbon dioxide, electricity usage, incoming solar energy, water collection, outside temperature, and wind speed. We may also be able to visualise some of the ebbs and flows of people moving around the building, and even the volume of people talking around the feature installation.

We are really excited to see the user experience of this building and The Heart at its centre come to life, since defining and delivering a consistent user experience has been a key focus for this building.

Amazing. But The Heart will be different from a dashboard for data analysis, Robert?

Correct. We have some amazing colleagues in the University who have developed some cool ways of visualizing data from buildings, but this project is more about interpreting that data differently. The Heart is not a tool for analysis, rather it is intended to create in the viewer a sense of a living being that is almost alien. We won’t always know why it’s behaving in the way that it does.

It is a bit like how we don’t always know what’s going on for anyone that you meet in life. Maybe their heart beats a bit higher and they might be breathing a bit heavily. There may be a lot going on inside.

Will anything happen at night?

At night, The Heart will dream. The artwork’s AI uses ‘dreaming’ to remember and forget the details of each day and compare them. And so when you are walking down Swanston Street at night, hopefully you will be able to look in and see it dreaming.

Has there been anything like this done elsewhere around the world, Lindsay?

This is very much a first of its kind, we feel. In our early research, we found a number of applications of data visualisation from a single input but not from multiple inputs and live abstract visualisation.

What a special project this is!

[Robert] Yes!

“We can only do this project that combines data, machine learning, artificial intelligence in a really cool, and unprecedented way here and now because of MDAP’s expertise. MDAP allows us to think big, about big, complicated projects that might have a big impact on people and the world. We need MDAP and Zaher!”

Alright, let’s wrap things up with your last thoughts on things you have each learned so far. Let’s start with you, Lindsay.

I’ve been pleasantly surprised by how useful the combination of artistic and digital development thinking has been. User experience is central to good technology design, and the artistic mindset looks to connect the experience to users in deeper, more meaningful ways.

Zaher?

I’m interested in arts, and I am always looking for ways to apply my skills in digital arts! I love finding new ways to implement my skills in new and unusual ways.

 And, what about you, Robert?

One of the core challenges with this project is about showing what we don’t normally see. To be more precise it’s about what we can’t see because it’s too small or too fast. To feel data requires senses that our bodies don’t have, so the digital world is always experienced in translation. Our artwork translates what a building feels into a register our bodies can sense.

Can we sense ourselves in our environments, buildings, and colleagues, and acknowledge their impact inside of ourselves? Can we foster greater empathy and awareness for what we might otherwise take for granted? If we can, then Melbourne Connect will be a great success.

“I think it is one of the big questions of our times: Can we participate in the digital revolution with all our senses, in a way that our bodies can grasp?

Thank you, all!

[Robert] Thank you, Jo! We are really excited to realise this project.

Lead image: ‘The Heart’ of Melbourne Connect will beat with a pulse extrapolated from data generated from sensors inside and outside the building prompting visitors to imagine the building as a living entity. Image: Robert Walton / Melbourne Connect