Futurlstic technology that is incorporated in everyday life

Walking in the streets will become fun with "urawaza"

When you read a guidebook such as the Michelin Guide, a place can look different - it's as if the streets have been given another layer. The Michelin ranks the meals and class of the restaurants through its stars. What we are concerned with is not class, but the augmentation of reality. (Kawada makes a pun with "class" and "augmentation" which are both pronounced "kakucho" in Japanese.) Adding that kind of value will make it fun to walk through the streets.

Similar to the Michelin Guide, I would like to make an "urawaza" (secret trick) guide for Roppongi - like what used to be in the game magazine Famitsu. People who don't know about the urazawa will walk just by; they will only notice that something a bit strange happened. Only the people who know about them will be able to manipulate things. They will suddenly become the privileged class.

If urazawa were sprinkled in the streets, I think people would want to find them; tourists will also probably walk all over the place, eager to try out the different urazawa. The urawaza guide could be placed on the guidebook racks in buildings such as Roppongi Hills and Tokyo Midtown. Discovering them will be the start of an adventure.

HUAWEI presents Starry Sky Illumination

An illumination show sponsored by Huawei that was held at the Tokyo City View Roppongi Hills observation deck held until January 29, 2017. In addition to the real night sky, visitors enjoyed the virtual sky by watching the "Touch the Stars Theater" created by Kawada as well as a universe simulator and an omnidirectional panorama provided by the Megastar planetarium.

The equivalent of the Konami Code in the city

Cities seem to be one-sided places - they just offer convenience. People come to cities for dates and for sightseeing and for work, simply to use the functions there. That's why I think it would be good to have points of interaction for city users and dwellers.

It would be fun to use urawaza with things that already exist in the town such as elevator buttons. There's the Konami Code where you push the video game buttons in a certain sequence like "up, up, down, down, left, right, left right, B, A". A similar sequence could be designed for elevator buttons so that when buttons are pushed in a certain order, the elevator suddenly turns into a dance hall. In places where there are no buttons, people could use their smart phones to do things like changing the color of ornamental lights or changing the background music.

It would also be fun to involve other people - getting people such as the staff at art museums to do things for you. In TV dramas, there are often scenes where a bartender sleekly brings a drink to the bar counter, saying "This is from the person over there." It would be good if we could make such subtle moves possible.

Media art that is used by people

Of course, it's a good thing for people to go out and look at art, but now that media art is maturing, I think it's about time for it to become more integrated in our daily lives. I think we should have functioning urawaza in the streets. People in the know will be able to use them - it would give the user the feeling of being a hacker or being part of some scheme.

It would be great if we could have such urazawa all over the country, don't you think- People will be telling each other, "If you go to such and such a place, you will be able to do such and such a thing." I've had this video game-like idea ever since I was a child.

Secret tricks at the "Touching Stars Theater"

We've actually made urawaza for the "Touch the Stars Theater" that's currently being shown. For instance, when you push the button with the sushi emoji（🍣）, the words "Eat sushi!" appears on the screen and the melody of the "Sushi kuine!" song is played. When you push the buttons with the hot springs emoji（♨）you get the song "Baban baban banban". With the bomb emoji（💣） you get "Dokan to ippatsu", and when you push the ghost emjoi（👻） and then Q, you get "Anone Q-taro wa ne." The song "Shonen jidai (Boyhood)" by Yosui Inoue comes on when you push the sunglass emjoi（🎐） and then the glass bell emjoi.

Touching Stars Theater

An interactive projection mapping that was projected on the 16 windows of the Tokyo City View observation deck. Visitors used their smartphones to change the brightness of the stars, the waxing and waning of the moon and to send "Wish upon a star" messages. Scenes from the anime movie "Your Name" were also projected.

I've made about 100 urawaza and I can hardly remember all of them. (laughs) Earlier, I made urazawa called "staff-pleasing buttons" where names of the staffers would appear in flashy way. I also made codes called "sponsor-pleasing buttons" that would play commercials.

The urawaza that we developers make are usually designed for a limited number of people. Of course we make them because they are entertaining, but they also serve as debug features, and are useful for checking that the programs are working properly. Unless we tell people about them, it's almost impossible for anyone to find the urawaza. We just give some essential hints and let people find the rest of the urawaza themselves. We can't give a full explanation because the tricks are supposed to be secret.

Roppongi is almost like a place of miracles

Roppongi seems to be a place with a lot of arrow signs - constantly filled with lights. People come to bathe in the lights or look at the lights. That is what makes Roppongi very different from other places. Incidentally, I used to host a J-WAVE program, so I used to come to Roppongi every week, and I often held live performances at Super Deluxe.

We had some very lively nights at Super Deluxe; we would play Morning Musume's "Koi no Dance Site" and I would wear a bust-shaped AR device that I created which emitted beams of light when we came to the lyrics "Sexy beam!" Then one night, I learnt that staff for the Hello! Project (a female idol group) were coming to the club, and I was thinking, "Oh no! We're going to get kicked out of this industry! This is the end of the AR Three Brothers!" But it turned out that they liked our performance; we were given an opportunity to make a debut commercial for the group S/mileage and we even got to perform in it.

After that happened, I really felt that Roppongi and I are compatible. I've been given the chance to do the "Touching Stars Theater" twice. Good things happen when I come to Roppongi - it's almost like a place of miracles. Maybe that's because six is the multiple of three. (laughs) (Kawada was referring to the three in Three Brothers and the six in Roppongi which literally means six trees.)

An experience that is determined by time and place

There are nice places in Roppongi like the art museum on the 52nd floor and airy spaces like Tokyo City View. The appealing thing about Roppongi is that the boundaries are blurred and it's not as if you're being told, "The art space starts here," or "The public space begins from here." I think the fantastical world of "Touching Stars Theater" works because it's in a place where the night view is lovely in the winter. If the venue had not been Roppongi, I probably would not have come up with the idea.

With the "Touching Stars Theater", we replicated all the actual stars. The entire space around the earth has been replicated as a celestial sphere; the design is as the sky would look like if the visitor were looking up from the window on the eastern side of Roppongi Hills. Since the lights in the streets are too bright, we cannot normally see beyond second magnitude stars. We've designed it to show the stars up to 12th magnitude - it's like turning up the lights in a room to make things visible. The earth is moving and so the constellations we see in the winter are different from those in the summer. We've also made fast-forward and rewind functions so that people can reset it to certain times. It's a show where the appearance of the sky is determined by time and place, and you never see the same sky twice.

We used a reduced scale for the size of the planets such as the sun and moon, and everything is based on scientifically proved data. People might think it's simply a fun show that allows you to send sushi messages, but actually we were quite serious when we made this. When you change the default setting, you can customize the show for any country in the world; for example, in Paris, visitors will be able to see the Parisian sky. It's such a genius idea that no one outside of Japan has yet asked us about it. We're ahead of our time; once we get understanding and recognition we'll consider taking the show abroad. (laughs)

Augmented reality is like a monster

I've lived in Chofu for a long time. I decided to live there because I love Shigeru Mizuki-san. It's a nice place. It's a city at the edge of Tokyo, and has lots of greenery and many slopes that are perhaps not on the maps. It's a place that gives you the feeling that monsters might really appear if you're not careful. That must be reason that Mizuki-san lived there; there is this "monster vibe" that makes you feel that something weird might happen.

[The manga character] Kitaro is an honorary citizen, and on the anniversary of Mizuki-san's death, the employees at the municipal office wear Kitaro costumes. I've also seen a middle-aged woman strutting in the shopping mall dressed like Bucciarati - the character in Part 5 of "JoJo's Bizarre Adventure". You could say that Chofu is a city with many mysteries. (laugh)

In my view, AR is like a monster. Stories about zashiki warashi (child goblin) and karaksasa kozo (umbrella-shaped ghost) have their roots in things such as natural phenomena or folklore. I think monsters served a purpose of reminding people to be cautious - like keeping the windows closed during cold winter to keep the cold out or forbidding children to walk at night because it is dangerous. The monsters were made up to convey these messages. They are really like the personification of warnings.

Monsters are the manifestation of something fearful - something that Junji Inagawas-san might describe as "Oh dear, how dreadful" when he's telling a ghost story. There is a reason for our fear, but that dreaded something has not appeared yet and we can't hear or touch it. I think there's a similar notion with AR in that you are working to make visible something that exists but is invisible.

Taste and smell will one day come to be replicated

In the previous"creator interview" , Kinya Tagawa-san noted how we are now living in very visually-oriented times; he said that the technology will eventually be able to deceive the human senses in the order of sight, hearing, touch, smell and taste.

Central to media devices is the input and output of visual and audio data. The smartphone is a device that people now carry with them all the time, but the surface of the smartphone is cold, expressionless glass. While people are emotionally attached to their smartphones there is something lacking in the appearance of the device - there is a discrepancy. I think such discrepancies that we feel in real life accumulate in our subconscious mind and lead us to have nightmares. We might dream of running on soft ground that makes us topple, or we might be repeatedly trying to punch something with wobbly fists. The senses that we have weird dreams about are the senses that can be artificially replicated.

In other words, the most replicable senses are in this order: sight, hearing, touch, smell and taste: it's the order of biological evolution reversed. It's said that you can tell whether you're dreaming or not by pinching your cheek, but I think that up until a certain era, people must have really felt the pain in their dreams. What I mean to say is that because we now live in a visually-oriented society, our dreams are visually-oriented, but in the past, dreams might have been about touch. Clay figures were probably made as mediums for touch and memory. They weren't meant made for visual appreciation, but for people to touch with their fingers and to understand. When I see clay figures in the glass cases of museums, I feel that they were the forerunners of smart phones.

Right now technology is still focused on the senses of sight and hearing AR and VR (virtual reality). But in the near future, there will be technology for recording and reproducing taste and smell and a more primitive reality.

A device to record the smells of food

A smell sensor for professional use - one that chefs might use - has already been invented; it can discern thousands of smells. If the smartphone were equipped with that sensor, then people could record the smells of food they like while taking pictures.

Food manufacturers already have the technology to transform smell into taste; it's possible to reproduce flavors by using chemical seasoning and fragrances. A sensor can record odor data of food and change that into taste data - forming a recipe like a RGB color code. It is technically possible, even now, to make something like furikake (dry seasoning).

So perhaps in the future, people will be banned from recording the smell of food. Returning to the topic of monsters, we will probably see the appearance of new monsters like the Bluetooth monster or the Wi-Fi girl in the style of the Cat girl. If you notice blue tooth marks on something, you will know the culprit is the Bluetooth monster. (laughs)

An application that will record data on objects

I'm interested in the twin-lens reflex camera and the infrared sensor or 3D sensor which will soon become small enough to carry around. Without using a measuring equipment, you will be able to measure things in an instant, so that will lead to so many possibilities.

AR used to be about creating a virtual space that was based on the two-dimensional, but this new sensor will allow us to create more accurate layers for spaces and architecture. AR Three Brothers had already made application for storing memories in objects. For example, sitting here now, I could take a photo of someone by my side and set the marker on the paper cup in front of me. The paper cup would then become a recording medium, and when I place my smartphone over it, the image of the person will be appear on the screen.

You could take images like this and give the object to someone as a present - it would be like a gift of memories. At the moment, the application is based on 2D technology, so the smartphone won't be able to discern this particular paper cup from another identical one, but that will become possible with the 3D sensor.

3D image of a moving public artwork at Roppongi Hills

In the year 2017, I want to really concentrate on developing a system along these lines and establish a de facto standard. One of the systems I hope to make is a donation system for rebuilding the stone wall at Kumamoto Castle which was damaged by the earthquake. I want to make something that will let people get data by placing their smartphones over the broken stone wall; the data will let them know how much has been donated and how many years it will take for the wall to be restored if donations keep coming at the current rate. The restoration plan and the future image will be made visible, so that people can see how much more could be restored if they make more donations.

This technology can be applied to all kinds of spaces and structures; it would be wonderful to do a project in Roppongi. Perhaps we could make something to guide tourists from abroad around the streets, and also incorporate urawaza. Perhaps we could create a moving 3D image of that spider (the public artwork "Maman") and tell a story of the elevator at Roppongi Hills being operated by the spider. It would be a modern version of "The Spider's Thread."

Our recent attempts include making works inspired by works of literature; we're trying to give a modern twist to the worldview presented in novels by using the power of technology. Last year, we made an AR/VR work based on Kobo Abe's "The Box Man." Famous works of literatures are known the world over, so I'm sure that a work based on "The Spider's Thread" would delight everyone.

AR Box Man

A work equipped with the technologies of VR and AR which was inspired by Kobo Abe's novel "The Box Man". In the novel, a man wears a cardboard box over his head, covering his body, and walks through the streets peering through a hole in the box. The work was first shown at the "Sense of Motion" exhibition held in Tokyo's Aoyama in November 2016.

A wireless LAN function in grandad's false teeth

As you know, in wearable computing, computer chips are inserted in things that you normally wear. I think that false teeth are perfect for that. For example, grandad's false teeth could be given a wireless LAN function. In a little while, we will have technology that will allow us to generate and store electricity with body heat. That means that grandad will eventually be able to become a "free spot" (Wireless LAN access point). Young people might gather around grandad and marvel that their smartphones are connecting smoothly; it might lead to a romantic age-gap relationship. (laughs)

I don't think that you necessarily need to show new things with a big pronouncement. People who want to throw high-velocity fastballs or a new type of pitch can do so, but our unit is more interested in doing down-to-earth things. To me, the really "futuristic" devices are those that can do remarkable things simply, even though the technology behind it might actually be intricate.

At elementary school, it was uncool to wear a new pair of "uwabaki" (slippers), wasn't it? Rather than things that are shiny and belong to a far-off future, I think it's much cooler to have things that can be normally used in our everyday lives.

Editor's thoughts
Kawada-san mentioned comments made by Kinya Tagawa-san, the co-founder of Takram. The dialogue that was held between Tagawa-san and Dominick Chen-san can be read on the site below. Incidentally, Dominick-san said that he thinks AI are monster-like things. （edit_kentaro inoue）