Month: October 2018

PComp Final: Sand God – Ideation

Posted on by admin

In my PComp final project, I wish to explore the following concepts:

  • How would AR impact the physical world?
  • How we can combine AR and AI to tell a compelling story?

In responding to these two concepts, I’m planning to create an interactive installation that allows the audience to defeat a physically presented monster by summoning an mask god existing in the AR world. The
audience will use drawings on the sand to summon and make attack / defend commands to the AR god, and apply damages to the monster. When the monster is eventually defeated after multiple successful attacks, the monster in the physical world will turn to a smiling face.

Below are the storyboard of the interactions between the audience and the
installation:

1. In the initial status, the audience will be facing an installation put on the table.

In the front, there will be the summoning stage: a big circular area
covered with sand, and a smaller circular area below it. The best bigger circle will be decorated with several tiny human figures holding their hands up, signifying it is a place to summon something. The smaller circle is painted with with mythical symbols, signifying it is a place to trigger the
summoning action.

On the right of the circle, there will be a pen-sized stick that looks like a wand of a wizard, sticking onto the installation. And this will be the audience’s tool to draw patterns onto the sand.

On the back of the installation, there will be wood pieces carved into the shape of a monster. And together, they set the stage of a fight between a tribe and the monster.

2. To begin, the audience will draw a pattern on the sand. There will be carved patterns (or maybe with instruction texts) on the installation to tell
audience what patterns to draw at the beginning. To draw the patter, the audience will pick up and hold the wand, and use the end of the wand to draw a pattern onto the sand. When the pattern is finished, the audience will hit the smaller circle with the wand, to signal a completed pattern.

3. A hidden camera will be used to take a picture of the pattern, and analyze it. If it matches the designed pattern to summon a god, a god will show up in the AR application.

4. To continue the fight to defeat the monster, the audience will continue to draw other patterns provided on the installation.

5. There will be mainly two categories of patterns – one for attack commands, and one for defense commands. Ideally, when the command is correct, the audience will be able to see the action of the god (to attack/defense) in the AR. The AR is also timed to draw tiny monsters to attack the summoned god.

And on the AR app, there will be statics showing the life of the god (which will decrease upon the monster’s attack), and the life of the monster. The
audience will continue to do so until the monster eventually defeated.

6. When finally the monster is defeated in AR, the monster on the installation will alter shape and change to a smiling face, meaning that the battle is over, and peace has arrived.

PCOMP Midterm – Part 2

Posted on by admin

This is the second post about my PCOMP Midterm Project – a ghost in a museum that will pop up if anyone walks closer to it, and subsequently tracking the motion of the person. To check with the first post about the ideation process and how the circuit is made, you can check here.

 

Room Setup

To find out how we should set up the projection at the Blue Room in Graduate Musical Theater Writing Program and make it work for the event, we went to the actual site and did a few rounds of testings to find out how we should position everything.

Things worth noting are that:

  1. When we’re using a room as a part of our project, the spatial layout of things in the room matters. For this project, it includes the size of the walls, the position of tables/chairs, the path that is created for the audience to navigate inside the room, and even the positions of the outlets (so that we can plug the projector and laptop to make the project works).
  2. The aesthetics of the project should go along with the environment in the room. In this case, since we’re creating for a museum of the 50s, we should make our projection feels like something from that time. For this, we’d changed the projected image and back ground for several times to make it really fit into the room, and we eventually decided to remove the LEDs in the eye artifact and put the sensors with a creepy feeling portrait instead.
  3. To create surprises (since we need to scare people for Halloween), we should projects in ways that are different and less common than people experience in everyday life. For instance we can alter the projection’s color (use black light to intentionally make the projection much less visible when we don’t want it), orientation (project it on people’s sides instead of right in front of them), and shape (map projections onto physical objects in the room that people would not notice at first glance).
  4. Taking advantage of sound in a room. It turned out to be much unexpected and creepier when I played the background sound from the other end of the room via a networked computer, instead of from the laptop right in front of the audience. According to the audience, it really felt like the room was occupied by the ghost.

 

Final Work

This is how our final project looks on the day of the Halloween event!

—- Projected Ghost Design—-

—- Triggering Artifacts (the portrait) —-

—-Projection on the Wall—-

PCOMP Midterm – Part 1

Posted on by admin

For the PComp midterm, I designed a haunted museum artifact for the Halloween Event hosted by the Graduate Music Theater Writing Program at Tisch. This is the first post of the project, and you can check with the finished project here.

Background

Our initial idea was to create a ghost on the wall that will surprise the museum visitors, and make the museum gourds talk upon visitors’ reaction. After further discussion with Briana, our coordinator from GMTWP, another team will be responsible for making the talking gourds, and we’ll focus on the wall ghost, and create other artifacts that can work with the ghost.

Since we’re using projections to make a ghost in the room, the room itself become the affordance of our project. It is both fun and mind-torturing to test out different combinations of item positioning to really take advantage of the space in the room. And this is very different from what I’ve been working on so far at ITP, where the scope of the projects is mostly constrained on a single piece of work sitting on the table.

Circuit Building

To make a projection that can will pop up unexpectedly,  we made a simple circuit includes a distance sensor that will send a triggering single to the laptop if someone places an object close enough to the sensor. Parallel to the sensor, we also put four groups of LEDs in series, in order to create an evil blinking eye artifact that will be triggered once the ghost is projected onto the wall.

Below are pictures and videos showing the building process of this artifact – the Eye of Cthulhu, from prototype to finish.

Visual Language – Week 6: Logos

Posted on by admin

7 weeks seemed long for someone who struggled to learn a new language from completely nothing to at least something; but 7 weeks also seemed short for someone who was amazed at how far he had come in the visual language world when looking back. Thanks to Professor Su Hyun Kim, it is the first time I realized that the speaking the language of the eye is so much fun and meaningful.

For our final week’s project, we were asked to design a business card for ourselves. For me, I made two sets of cards using a logo which represents my personality and resonates with the culture I was born in. And here they are:

 

Ideation

It sounded simple to just design a logo and put it onto a card. But to create a logo from scratch and make it really speak could take some head-aching hours.

I started from looking into elements that are unique about me, and have a deep connection to who I am. I’d thought of my face, my instrument, my favorite foods, my top ten movies and books.. But given the limited space on a small card, it turned out that a simple but convincing piece of element works the best. So I turned to using my name (Yuguang, or 宇光 in Chinese, which basically means the universe and the light) and created drawings based on its meaning.

 

Later, I realized that instead of drawing my own little pictures for each characters, the characters themselves have a figurative nature. They have gone through multiple iterations since B.C. 1600, and they actually evolved from figures people draw in describing their world. What’s shown below is the evolving history of the Chinese character of “light”.

I looked through these versions and eventually landed on the earliest and most primitive version – the Bone-Script Characters, the first one on the left.

 

Building Process

Based on the chosen character, I turned it into a logo, and simplified it so that it could possibly fit into a business card’s context.

What followed next were the topics we’ve learned: Typeface, Color and Composition. I tried several combinations of typefaces, palette and composition, and narrowed the results to two sets.

The hardest part was to find a typeface that somehow worked well with the logo, and a composition that could strike a balance between my name on the card and the logo. I reduced the colors on the first combination, and adjusted the composition and the orientation in the second to give the eye a better focus.

And here are the final designs:

 

 

ICM Week 5: Arrays

Posted on by admin

This week I made a motion tracking Match Man that can track your pose in front of a camera and turn it into a “stop-motion style” animation. This should be an ideal tool for any Kong Fu practitioner who wants to learn the Sonic Punch from the master but is too shy to ask in person. 😀

Here is what it feels like in Super Slow Motion mode:

Here’re the movements:

And here’s the guy who slowly did the Sonic Punch…

This project is built on top of a machine learning library called ML5. It is a super friendly JavaScript library which empowers anyone with even just a beginner-level coding skills to take advantage of the mighty AI and to do all kinds of stuff, from image classification to pitch detection. In this project, I used a model called PoseNet that can track the movement of a human body in front of an everyday web camera.

I also cooperated with Ellie Lin to create a Bread Man that can turn yourself into a piece of moving bread.  Check it out and have fun!

 

 

 

Visual Language – Week 5: Composition under PCOMP Theme

Posted on by admin

This week’s VL work is by far the most difficult one for me -not because composition is particularly hard, but because that I need need to come up with a holistic design with all the design principles we’ve learned so far working together. It is also the first time I genuinely realized how much it takes to give birth to a seemingly simple but beautiful piece of visual arts.

Below is my design for the ITP Winter Show 2018:

And now I’ll explain how it is made 😀

 

ITP Winter Show Poster

—-Version 1—-

In my first attempt, I began at conceiving how should I respond to the creative brief which says “we are hoping to see a more humanistic view of ITP – not just breadboards, hamburgers and LEDs”.

It was Oct 6, a Saturday morning, and also the date of NYC Comic Con. I was sitting at the ITP Shop located at 4th floor of Tisch, surrounded by cardboard, wires, wrenches, resistors, and the humming of laser cutting machines. While regretting that I should have bought the Comic Con tickets before they were sold out, an idea popped – how about turning the electronic circuit parts , the least “humanistic objects” around us, into human form? Even though I couldn’t go to the actual Comic Con, I could make a Comic Con of my own! With what I had in hand, I came up with these –  a wired & sensor version of Spider Man and an Iron Man.

Okay, I know they look kind of silly.. But at least they made Winner (a friend and classmate from ITP) laughed! And once I finished, I realized that I was merely trying to replicate the form of an existing figure. It was fun during the process, but as a visual design, it was not convincing, and it did not convey a story. I tried to make it better by destroying Iron Man and giving Spider Man some context, like this:

Still, not very convincing. I couldn’t explain why would Spider Man was dangling outside someone else’s terrace. Recalling what Christoph Niemann did in humanizing the objects, I thought I could take further advantage of the physical shapes of the circuit parts themselves, rather than just making them “look like” human. So I came up with these:

I turned to Winnie for opinions, and she told me that they were better and more interesting. But unless we were design for a Tech-Kitchen Conference, or a Cyber-punk Tennis Tournament, simply taking advantage of the physical forms of an object is not enough for my task. After all, I am designing for the ITP Winter Show!

And that was when I realized, I should have a guiding theme, or a story, in mind at the first place, before I actually stared working on the design. The message we are trying to convey should be the most important thing. And this gave me inspirations for my version 2 design.

—-Version 2—-

Considering what expresses a Winter feeling, I came up with an idea to team up my little FSR (Force Sensitive Resistor) guy with a snowman. After several experiments, I found that I could incorporate the poster design from Totoro, a famous Japanese animation movie,  to my story. After playing with more wires, I had the following draft:

Now finally, I thought, I was on the right track. So I followed this idea, and tried to make the composition better. I also moved it onto a black surface (so that the white wires would stand out) and provided it with abundant lighting.

Here what the second version eventually looks like:

At this point, I thought I might have finished my work. It should be telling a story; it was using the Rule of Thirds and scaling in the composition; and it the red LED nose should be able to give the eye a focus at the beginning. But, something seemed like missing.. I talked to Winner again, and she gave her feedback – It was a story, but quite inclusive. The characters in image live in their own world, and there were no conversations happening between the FSR guy, and me. So I decided to make a third attempt.

—-Version 3—-

With this “tech-and-human conversation” concept in mind, I suddenly feel that things could be a lot easier. I mean, making wire arts was fun, but it took a pain in the back and neck in order to carefully aligning the wires well together with shaky hand movements. Now, all I need to do is put human – myself – into the story! Like this:

Winnie also told me that the breadboard reminded her of a checkerboard. So I make some chess pieces with wires as well:

After some editing on Photoshop, they became two simple but more compelling posters. And I chose the first one as my final work.

 

Reflections on both VL and PCOMP

It took me four days to complete the poster design from scratch to finish. And you know what, the last attempt only took less than 3 hours! I think this gave a huge lesson in that designing visual graphics should follow the same “Fail Often, Fail Fast” guidelines just as coding or compiling a physical project. Since the actual execution of visual design could take much less time than writing codes or building circuits (since there are less debugging activities involved), it is very tempting to just rush to execution part first without much thorough thinking. And this could be detrimental to a visual design! Winnie also told me that it’s not uncommon that it takes a visual artist 10 hours to think and plan an idea, and only 10 minutes to execute it. Now I know what it means.

Another experience I gain was about PCOMP. During my making process of these posters, I was constantly asked whether I was working on a PCOMP project. And after hearing my “no” answer, they were surprised.  I think this somewhat reveals a certain degree of negligence of people, including myself, in the visual aspect of the circuits we’re using and building everyday. Indeed, they’re generally produced to serve a particular function – to digitize the physical world, to help lift an robotic arm, or to play a lovely melody upon a touch, but we should not forget that they, as physical objects, have a visual property, and we can take advantage of this as well in expressing ourselves.

 

Composition Exercises

And finally, these are my composition exercises.

 

 

PCOMP – Week 4: Analog Output

Posted on by admin

This week’s project is a continuation and the final piece of my “First Year @ITP Trilogy”. The name of the project is called ITP Life, and it’s a box that depicts the norm of many first year students in their first month at ITP.

It works like this:

 

Ideation

I got the project idea from my observation of a rotating servo. Unlike a motor which is capable of circling round and round in one direction, a servo can usually rotate only 180 degrees (some can do 360 max). So it comes with two directions.

It gives me a feeling of the progression and regression of a movement, and a symbolization of something that can go up and down. Relating to my first month’s experience at ITP, it reminds me of the work we have to do every week – as classes go by, the amount of work gradually piles up, and we keep working on them one by one until finally someday before the next class, they’re all cleared. Then the cycle begins again throughout the semester.

To express this “catch me if you can” feeling, I think it’ll be good to line up two servos vertically and in parallel, one stands for the homework, and the other stands for myself:

  • In order to enable two servos to rotate vertically, I need to build a base that can hold them both in the air;
  • Both servos will rotate in the same direction;
  • The homework servo will have a head start, and my servo will be the one chasing the homework;
  • The homework servo will gradually rotates at a constant speed, and my servo will rotate a certain degree as long as I send it a message to the micro-controller (meaning that I’ve finished my homework for a particular class).
  • To make the rotation of both servos visible, I’ll need something larger to delineate the servos’ movement. In this case, I’ll make two pieces of circular cardboard, and align their centers to those of the servos.
  • LEDs and paper with drawings can be attached to the cardboard circles to indicate which one is which.
  • When I finally catch up with the homework (the rotation of my servo equals to that of the homework servo), I’ll be happy. So, a joyful piece of music should be played for celebration (I put the Mario Level Complete music in it). Also the LED color would change.

 

Circuit Design

The circuit itself is pretty simple:

  • Two servos are connected to the Arduino board in parallel;
  • One red LED and one 220 ohm resistor are in series, and they’re parallel to the servos;
  • Same thing for a blue LED + resistor combo, and a RGB LED + resistor combo;
  • A buzzer connected parallel to the servos.

 

Building Process

The assembly of different parts were straight forward, as seen from the images below.

Here are a few things I found worth noting after the building:

  • The stability of connectivity between different components seemed to be susceptible to the rotation of servos. This had led to the unstable lighting of LEDs particularly. Without soldering, I could only attach extra tapes to glue them together.
  • With only a limited number of pins available at the Arduino board, the usage of pins should be carefully planned at the beginning. We should pay attention to components that would eat up a lot of pins (e.g. a RGB LED requires three Analog pins).
  • Also, the physical layout of pins should be mapped with the physical layout of components, so that the wires connecting the board and the components would not be a mess. In this case, I connected the two servos to the analog pins at both ends (pin 3 and pin 11), and reserved the analog pins in the middle for the RGB LED.

And that’s it! 😀

 

Code

/* ITP Life
by Yuguang Zhang @NYU ITP, Oct 3, 2018
*/

#include <Servo.h>
#include “pitches.h”

Servo servoITP; // the ITP homework servo
Servo servoMe; // my servo

int posITP = 70; // variable to store the initial servo position
int posMe = 0; // variable to store the initial servo position

int currentPosITP = 70; // current servo position
int currentPosMe = 0; //  current servo position

//Pins Setup
int ledITP = 2;
int buzzerPin = 8;
int ledCatchRed = 5;
int ledCatchGreen = 10;
int ledCatchBlue = 6;
int ledMe = 13;

// notes in the melody:
int melody[] = {
NOTE_G4, NOTE_C5, NOTE_E5, NOTE_G5, NOTE_C6, NOTE_E6, NOTE_G7, NOTE_E7,
NOTE_GS4, NOTE_C5, NOTE_DS5, NOTE_GS5, NOTE_C6, NOTE_DS6, NOTE_GS7, NOTE_DS7,
NOTE_AS4, NOTE_D5, NOTE_F5, NOTE_AS5, NOTE_D6, NOTE_F6, NOTE_AS7, NOTE_AS7, NOTE_AS7, NOTE_AS7, NOTE_C8
};

// note durations: 4 = quarter note, 8 = eighth note, etc.:
int noteDurations[] = {
8, 8, 8, 8, 8, 8, 9/3, 9/3,
8, 8, 8, 8, 8, 8, 9/3, 9/3,
8, 8, 8, 8, 8, 8, 9/3, 8, 8, 8, 4/3
};

void setup() {
pinMode(ledITP, OUTPUT); //ITP LED
pinMode(ledCatchRed, OUTPUT); //catchup Red LED pin
pinMode(ledCatchGreen, OUTPUT); //catchup Green LED pin
pinMode(ledCatchBlue, OUTPUT); //catchup Blue LED pin
pinMode(ledMe, OUTPUT); //my LED

Serial.begin(9600);

servoITP.attach(3);
servoMe.attach(11);
servoITP.write(posITP);
servoMe.write(posMe);
}

void loop() {

for (int i = posITP; i <= 150; i += 1) {

if (currentPosMe < currentPosITP) {

digitalWrite(ledITP, HIGH);
digitalWrite(ledMe, HIGH);

digitalWrite(ledCatchRed, LOW);
digitalWrite(ledCatchBlue, LOW);

servoITP.write(i);
delay(200);
currentPosITP = i;

if (Serial.available() > 0 && posMe <=160 ) {
String message = Serial.readStringUntil(‘\n’);
Serial.print(“received:”);
Serial.println(message);

if (message == “do homework”) {
for (int j = posMe; j <= posMe + 20; j +=1){
servoMe.write(j);
delay(15);
}
posMe += 20;
currentPosMe = posMe;
}
}

} else {
digitalWrite(ledITP, LOW);
digitalWrite(ledMe, LOW);

digitalWrite(ledCatchRed, HIGH);
digitalWrite(ledCatchBlue, HIGH);

// iterate over the notes of the melody:
for (int thisNote = 0; thisNote < 27; thisNote++) {
int noteDuration = 1000 / noteDurations[thisNote];
tone(8, melody[thisNote], noteDuration);
int pauseBetweenNotes = noteDuration * 1.30;
delay(pauseBetweenNotes);
noTone(8);
}

posMe = 0;
servoMe.write(posMe);
currentPosITP = 70;
currentPosMe = 0;
}
}

posMe = 0;
servoMe.write(posMe);
currentPosITP = 70;
currentPosMe = 0;
}

ICM & Visual Language – Week 4: “Palette of Me” Image Editor

Posted on by admin

This week we were asked to pick a palette of our own for the Visual Language class, and try to use functions and objects to draw a sketch with P5.js. I combined my works of both and made an image editor that can assign a 5-color palette to any given image. To begin with, I’ll start with the palette I’ve chosen.

Palette of Me
On a lovely Sunday afternoon of tons of sunshine, I wandered around the Rubin Museum of Arts with a friend and bumped into some magnificent pieces of Himalayan arts. As someone previously having some but not much exposure to Tibetan and Hindu culture, it was quite a fascinating and eye-opening visit.  Here are just a few examples of the beautiful pieces:
My favorite among them is this smiling deity portrait shown below, which in my opinion, skillfully stroke a balances between peacefulness and holiness. The color palette used here is classic in Himalayan arts: black, red, and orange with two complementary shades of brownish green.

Predictably, you might think that I would choose my palette based on this timeless combination. Well, I did plan to do that… BUT! As it is such a recurring theme in Himalayan arts, I decided to pick something completely different – colors that are nowhere possible to show up in any Himalayan pieces in the reality. After a few experiments, I finally came up with the following purple-based palette with a flashy green highlight.

Now that I have a palette, I would need an image editor to help me automatically transform photographs I love into this anti-classic theme. So I made my own editor to do the works.

P5 Image Editor

The idea of this editor is basically to compare the source image’s color, pixel by pixel, to each color in the palette. The color distance algorithms can be used to determine which palette color is “closest” to a given image color. After that, the editor will replace the color of a source image’s pixel with its closest color in the palette.  Tow color distance calculation algorithms are used in my editor – Euclidean, and CIE76. Here is the URL to my editor: YG’s 5-Color Palette Image Editor.

Final Works

Here are the final six compositions of my palette.  They are famous graffiti walls I visited in six different cities across the world: Guangzhou, Hongkong, Shanghai, Beijing,  Penang, and New York.

—- “Paletted Walls” —-

Added a bonus yellow: