Networked Interactive Wall Based on Multi-layered User Images Final project | GCT503 | Fall 2009 Si-hwa Park/Si-nae Ha/ Jieun Park/ Eunlip Lee
Introduction		An Interactive wall is often preferred an alternative digital media art because of interaction and communication between artwork on the wall and viewers in a public space. However, this interactive wall has limitations that viewers could just participate in this when they are in a specific room which this system installed and just share their experiences through a social networking platform. In this project, we suggest a networked interactive wall which connects separate walls with P2P system. At this wall, users could send people in different areas a mobile message and play a cooperative game by using a silhouette interface simultaneously. This makes remote partners communicate and interact in computer mediated communication area.
Project Design		Our project contains a character of interactive wall, networked interface and communication method. There need big three steps to proceed; abstracting silhouette of body, making interaction with applications on the wall and setting up network platform among the walls. 1) System Architecture The system construction starts to gain user's silhouette. The system consists of five elements: an infrared illuminator (IR), a camera (we use a webcam), a PC, a screen and a projector. We referred to Snibbe's Interactive installation architecture, but our system was transformed because we should capture not user's shadow reflected by IR but user's silhouette. This is because our cooperative game, a skipping-rope, should detect user's foot movement. To extract a whole body silhouette, we change a location of IR illuminators. 2) Software Silhouette extraction, network transmission & reception function through P2P, combination of the silhouettes altogether, interaction with contents, SMS mode, rope-jumping game mode are included in our application design. SMS mode: What we pursue in the SMS mode is for emphasizing the communication feature of the wall. In the SMS mode, people could send a message to the certain number appeared on the screen through their mobile phone. These messages appear on the upper part of the screen, on the randomly decided spot. When the body silhouette touches the message, the message disappears. All the messages on the screen will compose a rope in the next rope-jumping game mode. Rope-skipping mode: In the rope-skipping game mode, the rope consisting of messages starts to move up and down, turning round. The ropes located in different area start altogether at the same time if one player let this game start, even though there are no participants in the other locations. The other player could participate in the middle of this game and then all users standing on the different areas could play the game together. When the rope comes to the certain part of bottom, users should jump. If users could not jump on the right time and meet the rope, the program will detect and count it. If the counting number reaches ten times, then the rope disappears and the game is over.
Implementation & Technical Overview		1) User silhouette extraction: To make a user in front of the screen interact with a content of system, it is essential to extract user's silhouette. The total process for user silhouette extraction consists of infrared lighting and image processing. And for the image processing, we used 'Processing' and 'OpenCV' library for Processing which is an open source computer vision library. To extract the frontal silhouette of the user reliably, we direct invisible infrared light (using 2 consumer 850nm IR illuminators) on the screen behind the user. By using an infrared-pass filter tuned to this wavelength of infrared light, we can use an inexpensive black-and-white video camera to see only this infrared light. 2) Network system: To implement the network system, we used P2P architecture. To organize P2P system, we used 'JXTA' based on Java, which is an open network computing platform designed for peer-to-peer computing by way of providing the basic building blocks and services required to enable anything anywhere application connectivity. 3) SMS mode: To receive SMS, we used a SMS notification service of 'NateOn', a Korea messenger. When a user sends SMS to specified phone number for receiving message, NateOn, in which an owner of this number logged, notifies the message is arrived and shows that text. After SMS system receives a text message, it broadcasts this message to all interactive wall systems that are connected through P2P system. Each interactive wall system stores texts and draws it on the screen. Thus, all users of each interactive wall concurrently see the same SMS texts. A user also could delete a text message by using their silhouette through collision detection. When a text is deleted on any interactive wall, deletion information is broadcasted to other interactive walls. Thus, total interactive walls could manage the SMS contents with concurrence. 4) Rope-skipping mode: We used sine function to make texts turn round by periods like a skipping-rope. Each text that are lined up in a row move up and down according to the varied sine value by increasing the angle of sine without changing the their x-coordinate. And each text has logarithmically different maximum/minimum y-coordinate value to make a curve. For visual depth, a transparency of texts, in other words, the alpha value of texts changes according to the value of angle. 5) Collision detection: To detect a collision between a user silhouette and a text, we checked whether 4 vertices of the minimum rectangle including a text are in a silhouette polygon or not. We considered a collision is occurred if 2 more vertices are in a silhouette polygon.
Results		We suggest networked interactive wall system using multiple users' silhouette images. The most important features of the system are to enable communication among people in separate locations and feel correlation as if they are in a same room. In that point, we devised a whole body silhouette interface to enhance reality and protect people's privacy. With this system implementation, we install three interactive walls in remote rooms in a single building and users can send a mobile message to each wall, communicate with gesture and participate in a cooperative game, a rope-skipping.
Conclusion & Future works		Our design approach is to create a co-existing space in networked interactive wall and to select one of contents on a screen. We develop a shadow interface linked with corresponding remote user's images for interaction, and a design method of representing diverse reflections of communication partners in a row. At the next step, we'll develop diverse cooperative games and display them on a screen to increase user¡¯s selectivity. If two or three games play on a wall at a same time, users can choose their favorite ones. Moreover, we'd like to install this wall on numerous places, more than three, to check not only an effectiveness of this network system but a possibility of an alternative method in interpersonal communication system for at least a month. After this further experiment, we are applying our installation on a public space to use an information system by transmitting user's images in real-time.
references		Huang, J., Waldvogel, M. (2005). Interactive wallpaper, ACM SIGGRAPH 2005 Electronic art and animation catalog: 172-176 Suphawong, N. (2005). An interactive audiovisual installation & Performance. Department of Fine Arts in Design and Technology. Parsons The New School of Design Wesugi, S., Ishikawa, K., Suzuki, N. & Miwa, Y. (2004). Interactive spatial copy wall for embodied interaction in a virtual co-existing space. IEEE Lien, C., Wu, H., Chu, H. (2006). Interactive Care Wall. Workshop on pervasive display infrastructures, interfaces and applications Snibbe, S. & Raffle, H. (2009). Social immersive media: pursuing best practices for multi-user interactive camera/projector exhibits. CHI 2009 Bonanni, L., Xiao, X., Hockenberry, M., Subramani, P., Ishii, H. (2009). Wetpaint: Scraping Through Multi-layered Images. CHI 2009, p571~572 Davis, J., Bobick, A. (1998). A Robust Human-silhouette Extraction Technique for Interactive Virtual Environments, Proceedings of the International Workshop on Modeling and Motion Capture Techniques for Virtual Environments, p12~25 Processing, http://processing.org OpenCV, http://ubaa.net/shared/processing/opencv/ Kurose, J., Ross, K., (2005) Computer Networking: A Top-down Approach Featuring The Internet (3rd Edition), Addison Wesley, p75~78 JXTA, https://jxta.dev.java.net/ W. Randolph Franklin, PNPOLY ? Point Inclusion in Polygon Test, http://www.ecse.rpi.edu/Homepages/wrf/Research/Short_Notes/pnpoly.html Snibbe Interactive, http://www.snibbeinteractive.com/platforms/socialscreen/specifications/