Welcome to the Piaggio Fast Forward podcast. Join the conversation by subscribing to the PFF podcast at https://www.piaggiofastforward.com/podcast.
Welcome to Mobility + the PFF Podcast. I'm your host Jeffrey Schnapp, Chief Visionary Officer at Piaggio Fast Forward. It's a true pleasure to introduce today's guest Daniela Rus, who is director of the computer science and artificial intelligence laboratory at MIT, as well as Deputy Dean of Research for Schwarzman College of Computing also at MIT. A former MacArthur fellow, Daniela is a world renowned researcher in the field of robotics, artificial intelligence and data science.
The long-term aim of Daniela's pioneering work is enabling a future in which smart machines are fully integrated into the fabric of life, supporting people as they perform a wide array of cognitive and physical tasks. Her research addresses some of the gaps between the robots of today and a future characterized by pervasive robotics. Increasing the ability of machines to reason, learn and adapt to complex tasks in human centered environments, developing intuitive human robot interfaces and devising the tools for designing and fabricating robots swiftly and efficiently.
Welcome Daniela Rus to Mobility + the PFF Podcast. Daniela, in following your fascinating work at CSAIL the aspect that really has always captured my attention is your interest in the integration of robotics into everyday life. And I'd like to start our conversation by talking a little bit about what some of the areas of opportunity are that you see but also what the biggest challenges are to moving robots out of those domains, where they've had already a transformative impact, like on the factory floor but into these increasingly every day aspects of our lives.
So Jeffrey, I am very happy to be here with you to talk about one of my favorite subjects, which is how can we use machines to support people with physical and cognitive tasks? And this is an extraordinary point in the technological development because we are really on the cusp of seeing extraordinary advancements. If you think about the fact that 20 years ago, computation was a task reserved for the expert few because computers were large and expensive and difficult to deal with and now everyone computes. Now, it's extraordinary to think what comes next.
And to me, the question is, in this world so changed by computation that helps us with computing time work, what might it be with machines helping us with physical work? And I personally believe that we have extraordinary opportunities. One area that has been seeing very rapid progress is in autonomous driving.
And this is primarily because mobility of machines has made tremendous progress because of multiple factors. Because the hardware has gotten better and we have good sensors. The laser scanners have been transformational in how we think about mobility. I remember being a very young graduate student and using sonar to measure distances and none of the algorithms work because sonar was so noisy. But the minute when the laser scanner was introduced, everything changed. All the algorithms started working because we have reliable measurements.
So the body of the machine, the sensors, the actuators, how the body's constructed: this is tremendously important for advancement and for deployment in the physical world where we expect all kinds of unexpected situations to arise. On the other hand, the algorithms for mobility have progressed tremendously, we have map making, we have localization, we have a motion planning from point A to point B. We really have all the elements that enable a system to move in the world.
And so the question is, what are the opportunities? Where can we bring these systems that are much more robust on the hardware side and that are capable on the algorithmic and control side? There is so much potential for mobility and the potential is really in moving people and in moving goods. And I see these as two separate categories, let me talk about moving people first because there has been a lot of press on Robotaxi applications and companies are making huge strides in bringing autonomous driving to the roadways, but we are very far from level five autonomy.
And this is because the kind of situations that we expect on a public road where many vehicles are moving very fast all at the same time are still challenging for the level of algorithms that we have for the perception system of the vehicles and also for the control and planning system. Plus, our current solutions do not handle weather conditions. And so with Robotaxi, the current approaches will not work if there is torrential rain or if it snows.
And so we really need to improve these aspects of technology in order to get to the promise of a world with safe autonomous vehicles everywhere. In the meantime, I personally believe that the area of autonomous mobility that addresses, not Robotaxi, but addresses private roads where people move at slower rates on public roads with shuttles on campuses or in amusement parks or hotel properties or retirement communities. Those situations are much more amenable for autonomous solutions to moving people, for using robots to move people.
And I'm very excited about a future of mobility that is transformative in how we envision the use of a vehicle. Why is a vehicle only as a utilitarian vehicle from going from point A to point B? Why not use it as a mobile office that will give you a beautiful view as you conduct your meeting, or maybe it could be a private part where you can have a special dinner, a meeting or a personal dinner.
That does seem like a particularly exciting prospect that we could start to think our way out of the automobile as the vehicle of reference and start to imagine a whole mobile architecture, perhaps that moves with us to the places we need to be, we want to be, that are meaningful to us, and really sort of rethink the foundations of mobility, ecology, and our connection to place with the help of robots, right?
Absolutely. So I'm kind of imagining the revolving restaurants that sit on top of buildings as mobile pods, as glass domes on wheels that could give you an extraordinary dining experience, an extraordinary pleasurable experience as you move from one place to another. Commuting could be completely transformed if we rethink what a vehicle looks like. If we bring the sense of beauty in how one experiences the ride in the vehicle. And so that's on the moving people side.
Going back to your original question, there's also the opportunity of moving goods, and the opportunity of moving goods addresses logistics applications like the ones taken by Becker Robotics or sanitation applications, operations in a warehouse, operations to deliver food and to deliver other goods. And also operations that embrace the idea promoted by Piaggio Fast Forward which is to have the autonomous carrier that follows you and essentially carries what you need wherever you need to go.
These are all very, very exciting and frankly very realistic applications. You see the way I think about mobility and its readiness for commercial applications is in terms of how fast is the vehicle moving? How complex is the environment in which the vehicle has to negotiate its path and how complex are the interactions in that environment. And right now the technology is readily available for products that exist at the origin of these three axes of coordinates. Level five of autonomy is, of course, diagonally opposed to the origin and we can make our way over there but in the meantime there are many, many possibilities today.
Your work has been pioneering particularly in the domain of human machine collaboration. And of course, level five type autonomy would take us towards a world where in a sense the software is at the wheel, as you just stated, there's this whole open space between where we are right now in level five autonomy which involves a much more collaborative relationship between humans and the vehicles that they move with or in. Do you think level five is, aside from its technical feasibility, really desirable in terms of a kind of future society where robots would be woven into the fabric of human life?
The collaborative model seems to suggest almost a kind of new social pact if you like where humans and machines are constantly interacting and develop bonds, social bonds, bonds even of affection, much like the ones that connect us, not just between people but to pets or to the places that are meaningful to us. I wonder if maybe level five is something we ought not to obsess about and maybe shift our energies to really defining levels three and four much more richly in a much more textured way. I'm curious what you think about that idea.
Well, I kind of want to have it all really. I want the collaborations, I want the level four autonomy, I also want the level five autonomy for a variety of reasons. From a scientific point of view, we need to understand the science of autonomy and we need to enable engineering of autonomy. And so in studying autonomy, in building prototype devices, in building the mathematical foundations and the algorithms that allow us to create autonomous robots, we also understand ourselves in some way, and we get to push the boundaries on what we can engineer for.
And so I think that's very important and exciting. Now, will the people need it? Well, look, we get in trains, we get in planes, and in some sense the needs of people to move remains. What is extraordinarily exciting then could be enabled by robots is a higher degree of customization of the mobility process.
And so mobility on demand is what we started with when we began the project on autonomous vehicles in Singapore in 2010, and I believe that that remains an important paradigm. And just think about mobility on demand during the times of COVID, like how many times did you want to order food and you couldn't get on anybody's delivery list. I have had so much trouble with that but if I had a robot, I could simply send the robot to the store, somebody at the store who could put the goods I need in my robot and the robot would come right back to me and the food would all be safe.
So that's what I wanted to say about level five autonomy. I think it's important from the point of view of developing the science and understanding autonomy and intelligence. It's important from the point of view of engineering, more capable machines, and there are potential applications and products over there in the future. So I do want to go back to the collaboration idea that you started with because I think that is super important. In fact, I really love my car, I love driving my car, but I'm sometimes tired and make mistakes. The other day, I nearly had an accident and I would love to have my car act kind of like a safety check for me when I drive.
And in fact, as part of our project with Toyota, we are developing what we call a "guardian" autonomy system where the idea is that the car is a partner of the human, and the car has better understanding of the road because the sensors on the car can see further than the human eye and with respect to some aspects of the road, the sensors can make better estimates than we can with a human eye. And maybe through the use of communication technologies, the car could even connect with sensors installed at street corners or on other cars and know that there is somebody running around the corner, and that event is going to interfere with the trajectory of your car even before you have visual contact with that person.
So all of these ideas could make driving much safer. And this is one of the dreams behind autonomous driving, the dream is that we will have vehicles that will never be responsible for collision. But on top of that, what I would like is for the vehicle to become your friend, for the vehicle to get to know you, and to get to support you in a way that is possible even with some of today's approaches for modeling and personalization.
I think it's very important to also realize that today's technology is not well suited for making safety critical decisions, which is why having a collaboration between people and machines is important because people remain better than machines at so many things, but machines have become better than people in other ways. And so machines can move with greater precision, they can lift heavier objects, they can memorize and retain lots and lots of information, entire libraries, tomes are now at your fingertips in your phone. And so the question is how can we create a system that makes the most of both worlds of what people are good at and what machines are good at?
I think that's a fantastic point and I'm so glad you went back to this question of personalization and customization, because it's a topic that I wanted to bring up in our conversation. This relationship to your car as a kind of model, a car that understands your limitations, your cognitive limitations as an operator, as a driver, that can anticipate them, can compensate and correct for them. That kind of collaborative relationship, which you described even as almost like a friendship, a kind of agreement to work together. It's interesting because of course it's a friendship where the car doesn't become your cousin or your best friend, you don't hug it, it does what it does well, precisely as you were just describing, maybe better than you do. And you do what you do well perhaps better than this enhanced augmented automobile does.
That suggests a model of human machine interaction where machines are not striving to replicate the human, but they're striving to,in a sense, fulfill their greatest potential for performance for behavior. And that just brings me to a pet peeve that in the robotics conversation, at least outside of laboratories, there's so much focused on humanoid robots, the immediate impulses to think about robots in relation to replicating or even literally reproducing human behaviors.
And that's always seemed to me to be, if not a dead end, it's certainly not the path of where I see the most exciting, immediate possibilities. I very much share the vision you were just articulating around customization and human machine interaction and collaboration. But I'm less persuaded that robots should look like humans or try necessarily to replicate human operations. I'm just curious to hear your thoughts on the powers and limitations of that sort of humanoid impulse that has shaped the field of robotics. Going back to antiquity, really, because Hero of Alexandria, of course, dreamed of automata with the human form and was followed as such by Al-Jazari and by other pre-modern roboticists. So I'm interested in hearing your thoughts on that particular topic.
We have been inspired by the human form and we have had the dream of creating machines in our own shape that are smart and obedient for thousands of years as you pointed out. And I think that there is a lot of value that has come out of this line of thinking and will continue to come. I believe that by engineering machines that come in the human package, we get closer to understanding ourselves what makes us work and ultimately the science and engineering of intelligence and the science and engineering of autonomy remain grand challenges and will continue to be grand challenges for years to come.
So there is a very important aspect to doing that because in the process of making a machine that behaves in a certain way we generate hypotheses that neuroscientists could then take on and say, "Hey, I wonder if the brain does it in the same way." And then the research on understanding life, really, could take up those hypotheses and we could have a very productive iteration.
We have also had tremendous progress in industrial robotics for the past 60 years and you're right, most of the robots we have built today have been inspired by the human form, either in the shape of robot manipulators which have completely revolutionized industrial automation or in the form of humanoids. And then we have robots on wheels, so these would be the three categories of machines that we have had, we have built so far.
And I would say that we have come such a long way since the Unimate was introduced as the first robotic arm. And these industrial robots perform feats that humans can't even imagine being capable of. And yet, these robots remain isolated from people on the factory floor because they're heavy and dangerous to be around. And so the question I pose is how can we change that? How can we rethink what a robot is? Can we imagine robots that come in a wider variety of shapes and forms because the animal kingdom comes in a wide range of forms? We have so much biodiversity in nature.
And we've also created a built environment where the shapes are so different and we've created so many objects that are very useful and valuable. Now, imagine animating and roboticising some of the everyday objects around us and imagine creating an ecosystem of machines that are inspired by all kinds of shapes, the shapes in nature and also the shapes in the built environment.
And even going beyond that, imagine making machines that are made out of a wide range of materials, why make a robot just out of metal and hard plastic? Why not use softer materials? Why not use tissue? Why not use food or paper or even ice as the basis of making a robot? And so what I personally would like to do is to rethink what a robot is, both in terms of its form, its morphology, its function, what it can accomplish and the materials that it can be made of?
Precisely along the lines of imagining that kind of expanded vision of what a robot is and the kinds of skins that could be wrapped around a machine or the ways that a machine operates and intersects the everyday life experience of people. You're a leading person in the field of education and training of students at MIT in this field. What sort of skills or the ideal mix of skills for undertaking the kind of task you were just describing of re-imagining the robot itself and reimagining the forms that it assumes?
Well Jeffrey, here's what I believe. I believe that technological literacy is critical in the 21st century. So in other words, we need to expand what constitutes literacy and we need to go beyond reading, writing, and arithmetic to include technology. Now, what I mean by this is computational making and computational thinking, because those of us who know how to make things using the latest tools, those of us who understand the building blocks of making, and then those of us who can take what we make and breathe life into them through programming end up with super powers because we can make real, all the things we imagine and who wouldn’t want to do this.
Indeed. It sounds like a task for the 21st century but also I think the word superpowers is lovely because of course who wouldn't want superpowers, humans have spent their entire existence on this planet, dreaming of superpowers of one kind or another. So the ability to transform those dreams into forms of experiences is really a tremendously exciting horizon. I guess just a supplementary question has to do with what kinds of experiences aside from this core literacy that you were describing and computational making and thinking, which I think is absolutely crystal clear, would you want for somebody engaged in this activity, which is both a technical activity but it's also an imaginative activity?
I believe that learning the foundations, learning the basics is important. By learning physics principles, mathematics principles, programming principles, we get the skills to carry on and implement. But innovation rests in some sense on creativity, and creativity draws from so many aspects of work and life. And I also believe that it's equally important to know about art, to know about literature, to know about history, everything we know ends up being the basis upon which we can spring new ideas.
And so I am a very strong supporter for STEM education but that has to be done in parallel with the arts, in parallel with literature, in parallel with history and humanities in general.
And in terms of bringing in some of those disciplines that often exist at a certain remove from the STEM fields, how much do you think those disciplines themselves need to enter into the realm of computational making and thinking in order to understand better how to dialogue with some of these emergent opportunities and really exciting domains of innovation that are associated with a field like robotics?
There are many ways to make a contribution and express oneself. Contemplative thinking and writing is one way but even that is now supported and made more efficient by the use of computers as a means of writing and editing. Writing a book today is much more efficient, much faster than if you had to do it with pencil on paper. There are many ways in which we can contribute knowledge and we can be creative, and technology should be used as an enabler and technology can enable so many new activities and so many existing activities.
For instance, even the act of writing a novel which rests on contemplative thinking is encouraged and enabled by technology because writing your thoughts in a computer gives you much easier means of editing, polishing, reading, drafting. Looking beyond that we can imagine the use of technology as an enabler for art in so many other ways, the performing arts could be greatly enhanced by the use of technology. And in fact, here I have to tell you about a long-term collaboration I have had with a Pilobolus dance company where we were the first to create a robot human dance.
It was set to music by Schubert and it explored the story of a friendship between man and machine. And this piece was performed in New York and in Boston and in several festivals as early as 2010, 2011. And that was such a rewarding activity for us, it was so wonderful to bring technologists together with artists to get to a common language and to create something that's built on both of our skills and expertise.
And starting with that project, we then created something we call the umbrella project where we fabricated these electronic umbrellas whose color could be changed by pushing a button. And we gave these umbrellas to hundreds of people, put a camera above the ensemble of people and projected their collective image onto a screen. So each person became a colored pixel in this ensemble, and together people were able to create such beautiful images simply by dancing and by listening to high level commands from the MC of the event.
And this is the kind of participatory artistic experience that draws everyone in, that brings in people who are not professional dancers, but nevertheless people who have the ability to experience art and to participate in art. And so in starting from these examples, I can imagine so many other ways in which technology can bring new kinds of artistic experiences to people and technology in some sense could almost democratize the ability of someone to actively participate in an artistic event.
I'm very sympathetic with that vision and I think one of the aspects of it I find exciting is on the one hand that democratization, on the other the fact that these kinds of collaborations changed the way the dancers themselves thought about movement. Of course, dancers are experts in bodily movement but it's fundamentally different to move around the world, interacting with a mechanical agent of any kind, just as it is different to walk by yourself or walk in a group.
And as you know at Piaggio Fast Forward, the kind of complexities of that choreography are very much an object of some of the research we do trying to understand what is a meaningful and intuitive interaction between a robotic vehicle, and a human as the humans body becomes the interface, becomes the point of reference that in turn conditions, autonomous behaviors, may be very slight autonomous behaviors on the part of a robotic vehicle.
And the input goes the other way as well because as we observed the choreographed interactions between people and machines, we better understand what capabilities we need to develop for the machines so that interaction can become richer.
Indeed. And it's that bidirectionality where the creative juices start to flow and where unexpected and meaningful exchanges begin to occur, or relationships are forged, sometimes affective relationships. And I think you were hinting at that in some of your earlier answers regarding even the physicality of a robot. When the skin of a robot is a soft material, that's more skin-like and not hard metallic or plastic, obviously, our human predispositions to attribute value or to connect to that object change.
I know you've done a bunch of pioneering experiments in the area of using animal analogies to robots, I was wondering if, for our listeners, you could tell them about one or two of those instances.
So we've developed a robotic fish and this fish has very agile movement. This fish is able to do the escape maneuver, this is a very quick 180 degree turn which real fish do in order to escape the bigger fish who want to eat them. So we've been able to replicate that agility in our robotic fish. And we are very excited about the possibility of using our robotic device as a way of better understanding the coral reefs and more broadly, the oceans. Now, the coral reefs are like the canaries of the environment. And so by introducing a robotic device in a coral reef and a robotic device that can blend in, in some sense, and not cause too much noise, not disturb the water in unexpected ways.
The fish could be kind of a passive observant of the ecosystems and this robotic device can give us much higher quality information about what is happening in the coral reefs. I'm very excited about the possibility of extending this work as an element that will help clean the oceans because that's another big challenge. Another project we did very early on in 2009 was to use flying robots to observe closely the behavior of whales.
And so we had a really extraordinary experiment that we conducted with famed biologists and good friend of mine Roger Payne. We went to Argentina and we used our flying robots to get very close images of parts of whales interacting with each other during mating time and during birthing time. And these images were unprecedented. Our robots were able to fly very close up, they did not make noise, in fact Roger used to say, "Oh, they treat the robots the way we might treat the mosquitoes," just a little something buzzing above. So they were not disturbing of the activity of the whales and it was extraordinary.
And just between you and me, this project is turning into a very exciting new project about using machines to understand the language of whales. So imagine if we can use robotic fish and other external devices to really figure out what whales say to one another and to begin to develop the building blocks that would allow us people to talk with other species such as whales.
That's super exciting. I look forward to hearing more about that work as it unfolds. Moving towards a conclusion to our conversation today Daniela, I wanted to bring up a question, which is, I'm sure a question that you think about a lot in your role as a Deputy Dean of Research for Schwarzman College of Computing at MIT. And that is, a broader societal challenge we face in bringing more and more people into the STEM and computational fields and as well as specifically the field of robotics as a senior leader in that field and a woman...I'm wondering what your thoughts are with respect to engaging more women in STEM based fields but also obviously minority groups and other less represented groups?
Research is very clear for everyone. If you value innovation, diversity is critical. And so diversity is important in all aspects, in all aspects of the economy. And that means we have to take a very comprehensive look at how we train people and how we inspire them to join different career paths and to learn a wide range of skills from a very young age.
I believe that it all starts in schools. By the time the kids get to universities they've almost established their interests. And so I think that we have to inspire young boys and girls alike from all kinds of backgrounds, that technology is exciting. And as we look towards the future, I wish we could hear boys and girls alike say, "Hey, I want to be an astronaut, I want to be a climatologist, I want to be a computer scientist. I want to be a roboticist."
And that means that we have to take literacy of technology seriously. Now, if we do this, in the long run we will have a workforce that is prepared for all the new jobs that are coming. But in the short-term we also have to think about how we can help the existing workforce get re-skilled and trained to take advantage of the latest and greatest positions and with the latest and greatest technologies.
At MIT, we have been leading this conversation for about three years now. I have been hosting a symposium we call the AI and the future of work symposium, where the conversation is all about how new technologies are impacting the future of work and how we can bring these technologies to everybody. And we have formed a number of partnerships, for example, we have worked with a group in Kentucky, led by a startup company called Bit Source. This company is retraining coal miners into data miners and we have been using MIT technologies such as the App Inventor to help train the teachers who could then go back to their communities and train the people.
And I think that we've had success with that particular pilot program, I would love to see it applied more broadly because each community has different needs, and we really need the leaders of the community who understand the community needs to help connect with institutes of higher education with efforts where the tools for learning are being developed so that the knowledge could be propagated.
And then I would like to say that I also started a series of a TEDx MIT conversations, and the first one was dubbed Great Women Technologists of MIT. And this was about a year ago, the talks are available online, and they are extraordinary examples of the kinds of ideas that women at MIT are pursuing.
And this is in line with this notion that women have so much to bring to the table. They have so many ideas and creativity, and we would really like for girls and boys alike to educate themselves, to train themselves, to get to see what technology can do for them, but also what they can do for technology.
I think the timing of that initiative is great because as COVID and the pandemic have forced a lot of education online, the ability of teachers and schools to leverage those kinds of online resources to get young people excited about work in some of these sort of state-of-the-art fields increases. And I'd like to see a lot more outreach efforts coming from the major research universities towards the world of high schools and even middle schools and even elementary schools, because of course we can't always allow physical access to our research labs but to be able to communicate and to communicate in engaging ways, I think is a great start.
I am very passionate about AI in education and this could be a very profoundly impactful initiative at MIT and in other universities.
Well, congratulations on that initiative and thank you again Daniela Rus for joining us today on Mobility + the PFF Podcast.
Thank you, Jeffrey.
Thank you for listening to the Piaggio Fast Forward podcast and come back soon for further lively conversations about walking, light mobility, robots and the design of neighborhoods, cities, and towns. The PFF podcast is hosted by Jeffrey Schnapp, sound engineering by Robert Allen, narration by Ryan Harms, produced by Elizabeth Murphy, web designed by Jerry Ding. Intro music is Funkorama by Kevin MacLeod. End music is Your Call by Kevin MacLeod. Special thanks to Tory Leeming. To learn more about PFF and gita, please visit piaggiofastforward.com.