PCOMP and ICM Final – Week 14

Based on the results of the user testing, we made several modifications to the design of our project:

  • Feedback: currently, the audio interface happens at the front, while the tangible interface happens at the back. This creates a discrepancy between the two interactions, and we need to find a way to bridge them.
  • Adjustments: to allow the shift of attention smoother for the users, we decided to push the cage backwards (making it closer to the projection plane), so that physically they are aligned at the same place.

 

  • Feedback: the audio interface of the first step gives a command, while the one for the second step gives a suggestion. 
  • Adjustment: since the audio interface is intentionally kept for possibilities of more complex/interesting audio inputs in the future, we decided keep it, and we adjusted the audio instructions of the second steps to be commands as well.

 

  • Feedback: since the projected tweets are in green and red color, and they’re projected on the stairs (which looks rectangular), they confused people with the “green & red boxes” where the actual interaction should happen.
  • Adjustment: we removed the green and red colors of the projected tweets, and only keep the colors for the food containers of the cage. In this way, users should be less likely to be confused about what the “green food container” and “red food container” are referring to.

 

We also considered about removing the stairs and projecting the tweets on the wall directly. In this setup, we’ll put the cage on a stand, and put 2D bird cutouts or 3D bird models onto the wall at both sides of the cage, and project the tweets onto the wall by each bird. This design can remove the length restrictions imposed by the stairs, and it can give a larger room to the cage and making it the prominent element for interaction. We’ll do a round of testing to find out whether this new design works better than the old one.

PCOMP and ICM Final – Version 3

After collecting feedbacks for the second version of the bird cage device, we discovered the following issues regarding its design:

  1. The interaction between the users and the project is flat, as the device only takes in one user input (which is assigning topics to the cages) and respond only once, regardless of how fancy the response looks.
  2. The previous design makes a metaphor between tweets from Tweeter and the shape of bird cage, which make some sense. But this is mostly a connection based on the visual aspects of the two, rather than the tangible aspects – in other words, how a user is actually interacting with the physical form of a bird cage, the space that the cage occupies, and the bird inside the cage, etc., is not considered throughly enough.

After some discussion with Nick during the weekend, we decided to throw away the idea to make a physical representation of Twitter API by weighting how many people tweets on two given topics, and switch to focus on one of the actions that are most important for a person to own a bird cage – to put a bird into the cage. Based on this, we made a cardboard prototype to simulate a bird luring process. By throwing in hashtags of a topic on Twitter, we will lure an invisible bird that represents the Twitter data on this topic into the bird cage.

 

Following this design and a quick feedback from Tom, we further discussed what are the possible interactions that can both take advantage of people’s bird keeping behavior, and at the same time connects it to some intellectual activities that cannot be achieved by merely keeping a physical bird from the nature. I realized that since the bird is a representation of Twitter data, it is also a representation of the public’s opinion of a topic.

In the real world, by feeding the bird with some food and observing how the bird react to that food, we can know whether the bird likes it or not. In the same sense, if we feed the public data bird with a particular kind of thinking, by observing the bird’s reaction, we can have a physical glimpse into the public’s opinion or attitudes towards that thinking.

Further, I realized that this provides an opportunity for us to measure or rethink how our perceptions of social phenomenon are different from the larger public. Living in a physical environment, the opinions we possess are inevitably influenced by the people we engage with on a daily basis. For instance, the result of the recent midterm election in New York turned out to be quite different from a lot of people’s prediction at NYU. As someone who lives in a liberal community inside a liberal state, it is not uncommon that she/he’s gauging of the public’s attitude is somewhat biased. And this provides a good opportunity for us to reveal this bias through our feeding action with a public data bird.

So for version 3, we’re planning to create an invisible bird that allows users to guess and reflect on the guessing about public’s attitudes towards controversial topics on social media, as shown below:

PComp and ICM Final – Version 2

After collecting feedbacks from the lovely ITP community, I realized that the previous design about aligning AR with the physical world is a bit farfetched, in terms of the following aspects:

  1.  The starting point of the project is not self-explanatory enough. As urban people nowadays rarely play with sand, the “grab-wand-and-draw-pattern” action will need to be explicitly instructed, or printed out on the  project itself. And this somewhat makes the project not intuitive.
  2. The physical feedback of actions occurring in the AR world are missing, since the battle is visible only in the AR world. And the audience wouldn’t notice the changes in the physical world unless the monster is defeated in AR and then transforming into a physical smiling face. This somehow gives a feeling of “using AR for AR’s sake”. It is likely that the audience will merely sees it as an enhanced AR game.
  3. Since the audience’s commands are derived from continuous drawings on the sand, it requires them to clean up the previous pattern before a new one is drawn. The cleaning process can be a bit cumbersome, and the project does not provide a feedback of a “clean and ready” status. Also, the timing of cleaning and drawing can be tricky, since everyone may do it differently.

Given the above comments, I realized that emphasizing on the “interaction between moving AR objects and physical objects” is perhaps not a very good idea, or at least, it requires other forms of designs to make them working together really well.

After a discussion with my ITP classmate Nick Tanic, I realized that instead of focusing on the “movement relationship” between AR and physical world, maybe focusing on their differences in “visibility” is a better idea. As we know, physical objects are present all the time, while for AR objects, we still need an interface or a medium, like an AR-enabled phone, to actually see them. Since Nick is having an idea about using bird cages to visualize tweets, it rang my bell as it could become a wonderful stage for playing with this difference in visibility. So, we decided to collaborate on our finals, and. here comes the design: an AR-enabled bird cage system that visualize the quantitative differences in any given two topics/items/phenomenon and reveals trends of people’s topic selection.

 

Major Functional Elements

The concept of this installation originates from a idea about making a tool to measure social phenomenon. It will be comprised of four major parts:

  • a chandelier-like hanging structure
  • two empty bird cages that can move up and down
  • an application that is connected to the Twitter API and instructs the installation to move one bird cage up and the other down, behind the scene
  • an application that reveals what’s inside the two bird cages by using AR

 

How it works

The bird cage installation is made to measure social trends on twitter. The audience can come up with two hashtags they want to compare, (e.g. pizza vs burger, or Trump vs Hillary), and assign these hashtags to different cages. The up and down movements of the two bird cages will be synchronized with number of people who tweet about each topic. To see what’s being compared visually, the audience will pull out their phones and enable AR to see what’re inside the birdcages and being compared.

Since the audience will be doing two tasks: the topic assignment and the AR observation, it’ll be boring or less surprising for them to perform these two tasks in this order, since the assignment process somewhat gives away the AR contents. On the other hand, it will be interesting to reverse the sequence, and interpret it under a larger audience context. A possible workflow will be:

  1. We pick two topics and assign them to the bird cages;
  2. The cages move accordingly;
  3. Audience A comes to the bird cages, and see two imbalanced cages;
  4. Audience A pulls out his/her phone, and discover what are really being compared in AR;
  5. Audience A assign two new topics, and leave;
  6. The cages move accordingly;
  7. Audience B comes to the bird cages, and see two imbalanced cages;
  8. So on so forth.

And this creates an interaction among the installation and the audience flow over time.

 

PComp Final: Sand God – Ideation

In my PComp final project, I wish to explore the following concepts:

  • How would AR impact the physical world?
  • How we can combine AR and AI to tell a compelling story?

In responding to these two concepts, I’m planning to create an interactive installation that allows the audience to defeat a physically presented monster by summoning an mask god existing in the AR world. The
audience will use drawings on the sand to summon and make attack / defend commands to the AR god, and apply damages to the monster. When the monster is eventually defeated after multiple successful attacks, the monster in the physical world will turn to a smiling face.

Below are the storyboard of the interactions between the audience and the
installation:

1. In the initial status, the audience will be facing an installation put on the table.

In the front, there will be the summoning stage: a big circular area
covered with sand, and a smaller circular area below it. The best bigger circle will be decorated with several tiny human figures holding their hands up, signifying it is a place to summon something. The smaller circle is painted with with mythical symbols, signifying it is a place to trigger the
summoning action.

On the right of the circle, there will be a pen-sized stick that looks like a wand of a wizard, sticking onto the installation. And this will be the audience’s tool to draw patterns onto the sand.

On the back of the installation, there will be wood pieces carved into the shape of a monster. And together, they set the stage of a fight between a tribe and the monster.

2. To begin, the audience will draw a pattern on the sand. There will be carved patterns (or maybe with instruction texts) on the installation to tell
audience what patterns to draw at the beginning. To draw the patter, the audience will pick up and hold the wand, and use the end of the wand to draw a pattern onto the sand. When the pattern is finished, the audience will hit the smaller circle with the wand, to signal a completed pattern.

3. A hidden camera will be used to take a picture of the pattern, and analyze it. If it matches the designed pattern to summon a god, a god will show up in the AR application.

4. To continue the fight to defeat the monster, the audience will continue to draw other patterns provided on the installation.

5. There will be mainly two categories of patterns – one for attack commands, and one for defense commands. Ideally, when the command is correct, the audience will be able to see the action of the god (to attack/defense) in the AR. The AR is also timed to draw tiny monsters to attack the summoned god.

And on the AR app, there will be statics showing the life of the god (which will decrease upon the monster’s attack), and the life of the monster. The
audience will continue to do so until the monster eventually defeated.

6. When finally the monster is defeated in AR, the monster on the installation will alter shape and change to a smiling face, meaning that the battle is over, and peace has arrived.

PCOMP Midterm – Part 2

This is the second post about my PCOMP Midterm Project – a ghost in a museum that will pop up if anyone walks closer to it, and subsequently tracking the motion of the person. To check with the first post about the ideation process and how the circuit is made, you can check here.

 

Room Setup

To find out how we should set up the projection at the Blue Room in Graduate Musical Theater Writing Program and make it work for the event, we went to the actual site and did a few rounds of testings to find out how we should position everything.

Things worth noting are that:

  1. When we’re using a room as a part of our project, the spatial layout of things in the room matters. For this project, it includes the size of the walls, the position of tables/chairs, the path that is created for the audience to navigate inside the room, and even the positions of the outlets (so that we can plug the projector and laptop to make the project works).
  2. The aesthetics of the project should go along with the environment in the room. In this case, since we’re creating for a museum of the 50s, we should make our projection feels like something from that time. For this, we’d changed the projected image and back ground for several times to make it really fit into the room, and we eventually decided to remove the LEDs in the eye artifact and put the sensors with a creepy feeling portrait instead.
  3. To create surprises (since we need to scare people for Halloween), we should projects in ways that are different and less common than people experience in everyday life. For instance we can alter the projection’s color (use black light to intentionally make the projection much less visible when we don’t want it), orientation (project it on people’s sides instead of right in front of them), and shape (map projections onto physical objects in the room that people would not notice at first glance).
  4. Taking advantage of sound in a room. It turned out to be much unexpected and creepier when I played the background sound from the other end of the room via a networked computer, instead of from the laptop right in front of the audience. According to the audience, it really felt like the room was occupied by the ghost.

 

Final Work

This is how our final project looks on the day of the Halloween event!

—- Projected Ghost Design—-

—- Triggering Artifacts (the portrait) —-

—-Projection on the Wall—-

PCOMP Midterm – Part 1

For the PComp midterm, I collaborated with Abi Muñoz from ITP to design a haunted museum artifact for the Halloween Event hosted by the Graduate Music Theater Writing Program at Tisch. This is the first post of the project, and you can check with the finished project here.

 

Background

Our initial idea was to create a ghost on the wall that will surprise the museum visitors, and make the museum gourds talk upon visitors’ reaction. After further discussion with Briana, our coordinator from GMTWP, another team will be responsible for making the talking gourds, and we’ll focus on the wall ghost, and create other artifacts that can work with the ghost.

Since we’re using projections to make a ghost in the room, the room itself become the affordance of our project. It is both fun and mind-torturing to test out different combinations of item positioning to really take advantage of the space in the room. And this is very different from what I’ve been working on so far at ITP, where the scope of the projects is mostly constrained on a single piece of work sitting on the table.

 

Circuit Building

To make a projection that can will pop up unexpectedly,  we made a simple circuit includes a distance sensor that will send a triggering single to the laptop if someone places an object close enough to the sensor. Parallel to the sensor, we also put four groups of LEDs in series, in order to create an evil blinking eye artifact that will be triggered once the ghost is projected onto the wall.

Below are pictures and videos showing the building process of this artifact – the Eye of Cthulhu, from prototype to finish.

Visual Language – Week 5: Composition under PCOMP Theme

This week’s VL work is by far the most difficult one for me -not because composition is particularly hard, but because that I need need to come up with a holistic design with all the design principles we’ve learned so far working together. It is also the first time I genuinely realized how much it takes to give birth to a seemingly simple but beautiful piece of visual arts.

Below is my design for the ITP Winter Show 2018:

And now I’ll explain how it is made 😀

 

ITP Winter Show Poster

—-Version 1—-

In my first attempt, I began at conceiving how should I respond to the creative brief which says “we are hoping to see a more humanistic view of ITP – not just breadboards, hamburgers and LEDs”.

It was Oct 6, a Saturday morning, and also the date of NYC Comic Con. I was sitting at the ITP Shop located at 4th floor of Tisch, surrounded by cardboard, wires, wrenches, resistors, and the humming of laser cutting machines. While regretting that I should have bought the Comic Con tickets before they were sold out, an idea popped – how about turning the electronic circuit parts , the least “humanistic objects” around us, into human form? Even though I couldn’t go to the actual Comic Con, I could make a Comic Con of my own! With what I had in hand, I came up with these –  a wired & sensor version of Spider Man and an Iron Man.

Okay, I know they look kind of silly.. But at least they made Winner (a friend and classmate from ITP) laughed! And once I finished, I realized that I was merely trying to replicate the form of an existing figure. It was fun during the process, but as a visual design, it was not convincing, and it did not convey a story. I tried to make it better by destroying Iron Man and giving Spider Man some context, like this:

Still, not very convincing. I couldn’t explain why would Spider Man was dangling outside someone else’s terrace. Recalling what Christoph Niemann did in humanizing the objects, I thought I could take further advantage of the physical shapes of the circuit parts themselves, rather than just making them “look like” human. So I came up with these:

I turned to Winnie for opinions, and she told me that they were better and more interesting. But unless we were design for a Tech-Kitchen Conference, or a Cyber-punk Tennis Tournament, simply taking advantage of the physical forms of an object is not enough for my task. After all, I am designing for the ITP Winter Show!

And that was when I realized, I should have a guiding theme, or a story, in mind at the first place, before I actually stared working on the design. The message we are trying to convey should be the most important thing. And this gave me inspirations for my version 2 design.

—-Version 2—-

Considering what expresses a Winter feeling, I came up with an idea to team up my little FSR (Force Sensitive Resistor) guy with a snowman. After several experiments, I found that I could incorporate the poster design from Totoro, a famous Japanese animation movie,  to my story. After playing with more wires, I had the following draft:

Now finally, I thought, I was on the right track. So I followed this idea, and tried to make the composition better. I also moved it onto a black surface (so that the white wires would stand out) and provided it with abundant lighting.

Here what the second version eventually looks like:

At this point, I thought I might have finished my work. It should be telling a story; it was using the Rule of Thirds and scaling in the composition; and it the red LED nose should be able to give the eye a focus at the beginning. But, something seemed like missing.. I talked to Winner again, and she gave her feedback – It was a story, but quite inclusive. The characters in image live in their own world, and there were no conversations happening between the FSR guy, and me. So I decided to make a third attempt.

—-Version 3—-

With this “tech-and-human conversation” concept in mind, I suddenly feel that things could be a lot easier. I mean, making wire arts was fun, but it took a pain in the back and neck in order to carefully aligning the wires well together with shaky hand movements. Now, all I need to do is put human – myself – into the story! Like this:

Winnie also told me that the breadboard reminded her of a checkerboard. So I make some chess pieces with wires as well:

After some editing on Photoshop, they became two simple but more compelling posters. And I chose the first one as my final work.

 

Reflections on both VL and PCOMP

It took me four days to complete the poster design from scratch to finish. And you know what, the last attempt only took less than 3 hours! I think this gave a huge lesson in that designing visual graphics should follow the same “Fail Often, Fail Fast” guidelines just as coding or compiling a physical project. Since the actual execution of visual design could take much less time than writing codes or building circuits (since there are less debugging activities involved), it is very tempting to just rush to execution part first without much thorough thinking. And this could be detrimental to a visual design! Winnie also told me that it’s not uncommon that it takes a visual artist 10 hours to think and plan an idea, and only 10 minutes to execute it. Now I know what it means.

Another experience I gain was about PCOMP. During my making process of these posters, I was constantly asked whether I was working on a PCOMP project. And after hearing my “no” answer, they were surprised.  I think this somewhat reveals a certain degree of negligence of people, including myself, in the visual aspect of the circuits we’re using and building everyday. Indeed, they’re generally produced to serve a particular function – to digitize the physical world, to help lift an robotic arm, or to play a lovely melody upon a touch, but we should not forget that they, as physical objects, have a visual property, and we can take advantage of this as well in expressing ourselves.

 

Composition Exercises

And finally, these are my composition exercises.