PCOMP – Week 3: Observation & Analog Input

On Saturday, Sep 22nd, I volunteered at the World Maker Faire hosted at the NYC Hall of Science. It was a precious and wonderful experience – I was able to see tons of interactive projects, to engage with the bigger maker community, and to learn from some of the most innovative people in NYC.

Among all projects that I got a chance to visit, one impressed me the most – an Umbrella-sized Planetarium. It looks like this:

It did not have fancy LEDs blinking all over, from top to bottom; it did not have techie-feeling AR/VR scenes through which you can peak in to the future; it did not even produce a single sound. But in my opinion, it is a brilliant project, because:

  1. It cleverly adopts and integrates the conceptual model of an umbrella, where you would always hold it UP to keep the rains from falling onto you, and that of a planetarium, where you would always look UP for the stars. These shared UPWARD motions provide a clear and intuitive mapping for me to understand and use this mini-size planetarium.
  2. The handle of the umbrella provides an affordance to grip and turn it around. The blue straps are tying up the wires and they somewhat work as the signifier for gripping and holding the umbrella up. The monitor attached under the face of the umbrella is turned on and emitting bright light, which serves a signifier for the eye to look at. All possible actions are clear.
  3. The image on the monitor refreshes right after I pick the umbrella up. So the feedback is immediate. The star images shown on the monitor aligns with what we see in an actual planetarium. So the feedback is informative.
  4. And last but not least, the pattern on the umbrella looks good 😀

My observation on how people use it somewhat resonates with my own experience:

  1. When people saw this umbrella first (which is laying horizontally on the table), most of them would look straight into the monitor first (for about few seconds). Then they would put their hands around the handle, and held up the umbrella. The others would just held it up first and then looked to the screen. No one seemed to be confused about how to use it during my observation period.
  2. The fun part began when people started to turn the umbrella around. As long as the monitor image was synchronized smoothly and correspondingly with the portion of sky presented under the planetarium, people would happily pointing left and right, up and down to check the different stars. But once the image got delayed and did not sync with the movement of hands, people would seemed a little confused and tried to adjust the pointing direction of the umbrella, and to figure out where to point exactly. This little mess would generally take 2-5 seconds to resolve.
  3. The entire exploration of the umbrella planetarium went from 10 seconds to about a minute. Overall, I believe most people found it intuitive to use and had some fun exploring this tiny piece of starry sky.

 

Analog Input & Digital Output

I continued my series of reflection on my mental states this week (it’s called Fahrenheit Fright) by trying to resolve my frustration with the Fahrenheit system with a simple analog input and digital output system.

Here are the components:

With a temperature sensor and three LEDs in different color, I measured the Fahrenheit degrees in the environment, transformed them into the Celsius system, and categorized them into three discreet levels – levels that I use as guidelines for dressing up daily:

  • Blue LED lighting up with any temperature under 26 C =  It’s a bit cold;
  • Green LED lighting up between 26 – 32 C = It’s comfortable with one T-shirt;
  • Red LED lighting up above 32 C = It’s hot! No need to bring any long-sleeves.

Here are the outputs:

And there is how it works in actual environment:

 

Visual Language – Week 3: Typography

There are two tasks in this week:

1. Redesign an airline boarding pass
2. Create three expressive words

 

Resign the Boarding Pass

The given boarding pass shown above looks confusing and unreadable due to the following reasons:

1. The hierarchy of this boarding pass is a mess. The only thing we can clearly identify is the Seat and Zone information, which is outlined in a box. The bold, larger sized “DOCS-OK” text also stands out, but it doesn’t help in guiding a passenger to board his/her desired flight. Other than these, everything else appears to have the same weight. So identifying a particular piece of information seems pretty hard.

2. The typefaces are used inappropriately.
It is obvious that most of the Ten Type Commandments are violated: Texts are neither left-justified or aligning to one axis; Changes in weights is small and do not create a contrast; There appears to be at least 4 point sizes with little contrast; Information is not grouped and runs all over; And finally, the integrity of the typefaces seems to be broken

 

Here’s what I did to fix it:

1. Reconsidering the most important message a passenger needs to know in order to board the flight, and assign them the first priority visually
2. Build contrast between information using adequate weight and point size variance
3. Align and group information so that relevant things stay together
4. Create a balance between the graphic elements and text elements by adjusting their relative size and position

 

Three Expressive Words!

Since I began school at ITP, I was forced to pick up cooking since meal prices at NYC is intimidating. So, I chose three cooking verbs that I use most frequently to keep both myself and my wallet healthy. Here they are!

—-Slice—-

—-Fry—-

—-Scramble—-

 

Visual Language – Week 2: Signage

The assignment of this week is:

  1. Go outside and photograph 2 examples of unsuccessful signages and 2 examples of signages you like and post all 4 images to your blog.
  2. Choose one of your unsuccessful signs and redesign it.

And here’re my findings around Downtown Brooklyn.

 

Good Signages

—-1—-

The first good signage I found is a combination of signs at a construction site. They express warnings in an efficient way by placing the icons and text messages side by side, and assign colors that correspond to the context of the message.So at a first glance, we’ll know what is allowed, what is prohibited, and what we should do.

Less successful construction signs, in contract, don’t take advantage of the power of icons and images. In the above examples, the alerting “Danger” texts are given to express a feeling of warning. But we have to look closer to check the content of the supplementary texts to figure
out exactly what is dangerous here.

—-2—-

The second good signage is the instruction sign for traffic lights. I think it’s good because:
1. the images are intuitive and easy to understand;
2. the images used are consistent with the actual traffic lights
3. the three instructions are aligned in an order corresponding to the changing order of the lights;
4. the instructions begins from the Walk instruction, which is usually the most desirable action a pedestrian wants to take in crossing the road.

 

Unsuccessful Signages

—-1—-

The first unsuccessful design I found are those on a Chase ATM. It has a bulky touch screen and it looks fancy and techie. Yet as someone who has never used a Chase ATM and wants to make a deposit or withdrawal, it really took me a while to figure out what I should do first until I eventually spotted the card insertor. The touch screen is not helping because it displays a commercial. The blue LEDs highlighting the “Check In” function distracted me and made things even a little worse.

In contrast, another older type ATM I randomly ran into has a prominent instruction plate that captures my attention immediately. The images in the plate for each separate ATM function are consistent with the images of individual functions, and their arrangement mostly aligns with the
spatial layout of the individual functions on the machine. Indeed, it might have less functionality than the Chase ATM; but my mental burden of using it is a lot less than that of Chase’s.

—-2—-

The second not-that-successful design, in my opinion, was the signage for fire connections of Brooklyn Point. The instruction message of a connection is comprised of three parts: the type of the connection (sprinkler, standpipe, and combination of both), its applicable address, and
its applicable floors). It seems that there’s a correlation between the type and the color of connection, but it still takes a while for me understand. And also, the text messages have the same typeface and font color, and they are piling up upon each other, making it somewhat hard to identify the specific information I’m looking for. I tried to come up wit a better design below.

 

Redesign of the Fire Connection Signage

Due to time constraints, I was not able to make a complete and polished redesign. But here’s my hand sketch:

1. Separate the three types of information
2. Incorporate images to make it more intuitive
3. Align the colors of the connection to the color of the texts

ICM – Week 2: Variables

In this week, we’re asked to do create a sketch that includes:

  • One element controlled by the mouse.
  • One element that changes over time, independently of the mouse.
  • One element that is different every time you run the sketch.
  • e.g. Try refactoring your Week 1 HW by removing all the hard-coded numbers except for createCanvas(). Have some of the elements follow the mouse. Have some move independently. Have some move at random.
  • e.g. Do the above but change color, alpha, and/or strokeWeight instead of position.
  • Or do something completely different!

—- What I Draw —-

The self portrait I drew on Week 1 was a drummer. Now, incorporating the elements mentioned above, I turned my self into a DJ drummer lol!

  • Use the mouse to control daytime mode or night-club mode
  • Make myself shining in various colors over time
  • Incorporate the Left Arrow and Right Arrow keys to play the drum
  • Make myself BURNing in RED when I drum crazily with the keyboard!!

—-Link to My DJ ver. Portrait —-

URL: https://editor.p5js.org/full/Sk8ONr1Km

Have Fun!

PCOMP – Week 2: Electricity

The assignment this week is to come up with a simple application for switches and LED circuits. In responding to my desperate hunt for inspirations after school begins at ITP, I made a Inspiration Charger that will (hopefully) charge inspirational power and brilliant ideas into my pen, and thereby help me survive through the first semester.

 

Ideation

The Inspiration Charger idea is originated from Tom’s introduction to switches in class – different from a push button, a switch will stay at one static state if no action is performed on it. So I though it might be good to make a switch that can reflect my two states at ITP recently – one is excited and full of creative impulse; the other is exhausted and needs an energy refill.

This in turns calls for a charger of inspiration. With this magical device, I can put my pen in and charge it with the craziest ideas when I’m busy running everyday errands. Once I’m ready to work, I can take my pen out from the charger, and the innovative ideas will flow immediately from its tips.

 

Building Process

The application itself is fairly simple and straight forward. It is a serial circuit with a resistor, an LED, and a switch. The challenge comes when I need to turn the switch into a charger that can be triggered by the insertion and removal of a pen.

After checking on the materials in the ITP Shop, I found a metal spring that seems to be good for providing both the force to hold a pen and the consistent connectivity in a circuit. So I made a simple pen stand using a piece of cardboard, attached conductive paper to its back, and glued the spring underneath.

In this way, when a pen is inserted, it’ll break the connectivity by creating a gap between the two conductive paper tiles, thereby shutting off the “Inspiration On” light and charge the pen with magical inspiration power; when the pen is taken out, the conductive paper tiles will be pushed against each other by the spring, and the “Inspiration On” light will be turned on, symbolizing that the pen is now full of imaginary power.

 

Final Application

This is how it works eventually:

 

 

 

 

PCOMP – Week 1: Reflections on Interaction

This week’s readings include the first two chapters of Chris Crawford’s book The Art of Interactive Design and an article “A Brief Rant on the Future of Interaction Design” by Bret Victor. Both of them are insightful and thought-provoking pieces, and they led me to take an introspective look at my understanding about interaction and my working trajectories about interactivity. I’ll start with Crawford’s book first.

 

The Interactivity Formula

According to Crawford’s definition, interactivity is “a cyclic process in which two actors alternately listen, think, and speak”, and it has two implications: the Degrees of Interactivity, and what contributes to a high degree of interactivity.

The notion of Degrees of Interactivity reveals that it is “a continuous variable with relative measures”, rather than a “blank or white” conclusion. Upon reading about the refrigerator light example, I immediately thought about the mirror, especially the wooden mirror at the door front of ITP.

Personally, I love this mirror so much that I played with it for nearly 15 minutes on my first day at ITP. It is interactive by definition mentioned above, but at what degree? Since the quality of interactivity is gauged by the combined quality of the actors’ listening, thinking and speaking performance, is it at a somewhat lower-level interactivity since the mirror’s “thinking” was to merely reflect the my physical image regardless of however I move? And if it is, what can I do to increase its interactivity? A few things that came to my mind includes:

  1. Using a camera with higher-definition CMOS to capture detailed movements of my body (i.e. to listen better)
  2. Using a different algorithm to flip the wooden tiles, e.g. for ladies it would reflect a light image on top of a dark background, and for gentlemen it would do it reversely, or all tiles flip triple times if I raise my arms? (i.e. to think better)
  3. Using different flipping patterns (i.e. to speak better)

In terms of the examples of dancing and performance arts, I believe AI is really a good tool to boost up the interactivity. It can be used in areas such as allowing machines to generate music according to the dance (to enhance the “think” & “speak”), or providing a generalization of a large audiences’ responses for performing artists (to enhance the “listen”).

In a word, this is indeed a good framework for us to discuss about, think of, and design around interactivity. Now, let’s turn to Bret’s article.

 

The Spiritual and Physical Ramification of Interaction Design

Bret’s article is centered around two utterly amazing things that we mostly ignore in our designs – “Hands feel things, and hands manipulate things.” This is true, and it had led to the prevalence of Pictures Under Glass designs nowadays.

My thinking is that although Pictures Under Glass, as a transition technology, is indeed inept for physical interaction in terms of lacking tactile responses, it is however a cornerstone in guiding our “spiritual” interaction, or in other words, the sort of interaction that functions mostly at pure information-exchange level. One of its ultimate form I believe, will be implants into our eyes so that such “pictures” are rendered real-time in our vision and the interaction is triggered by our thoughts, or brainwaves. Just image I can check tomorrow’s weather by simply reading aloud “weather tomorrow” in my mind and see the weather image right in front of my vision. This should still be easier and more efficient than getting a response from a hand-held device which vibrates one time for sunny days and two times for rainy days, right?

Hands are invaluable for physical interactions. But we shouldn’t neglect the implications of Pictures Under Glass designs in guiding our spiritual interactions as well.

And these are my thoughts of the two readings.

ICM – Week 1: How Computation Applies to..

Well, this seems like a big topic at first glance, especially when computational power is ubiquitous nowadays and I am so easily attracted to new stuffs. Two things, however, rang my bell immediately, and they’re in fact part of the driving forces that have brought me to ITP.

 

AI drumming

(credit: http://www.melbournedjembe.com.au)

Being a drummer in West Africa is not easy. Unlike most modern performing art forms, where a dancer choreographs over the music, the style in West Africa is done somewhat reversely – a dancer improvises on the movements, and a drummer watches carefully and interprets the grooves out of those movements.

As someone who specializes at this kind of drumming – a djembe player, to be more specific – but isn’t equivocally adept at dance movements, I wish there is a sort of intelligence available that can help me unlock the dancers’ secrets.

And this is where computation comes into place. With Artificial Intelligence, we should be able to analyze the correspondence between the visual patterns in classic dance video footage and the underlying musical rhythms performed in those videos, by the most skillful and experienced drummers. Perhaps AI can also do the drumming as well, automatically producing the rhythms upon observation of human movements just like a master drummer.

 

Performing the Space

(credit: church stage design ideas)

Another interest of mine is to figure out how the 3D spaces we’re in during live performances can be transformed into part of the performing experience.

Up till today, artists still perform primarily on a stage – that is, for a typical 4 pieces band, they play music within a 20′ x 12′ box. Theaters offers bigger stages for artists to sing, dance, and occasionally, interact with the people sitting on the front or aisle seats. But the biggest proportion of that space, the audience’s space, is mostly unused (except for offering a place to seat). Immersive theater offers a full experience, but that space is tailored, somewhat fixed, and takes $$$ to build.

With abundant computational power, the ability to project augmented layers of information onto the actual environment, and with IOT enabling objects around us to know who we are and what we’re doing, it is possible that the untapped space aforementioned can be turned into part of the performance as well. Bearing no physical forms, virtual objects and structures can exist in any part of the performing space. What will a performance taking advantage of this look like? I’m eager to find out.

ICM – Week 1: Self Portrait

Introduction

The first week’s work for ICM is to code a self portrait with P5.js. Since drawing by calibrating the position of each pixel on the canvas is somewhat painful for me, I decided to look for an alternative way that can take both the advantage of the comfort and ease provided by hand sketches, and the ability to manipulate visual elements dynamically by coding.

Inspiration

(photo credit: kiritani 88)

A couple weeks ago, I saw by an art work created by Hirohiko Araki, which “Jojo-colored” a wall in shinjuku, Tokyo as shown below. (Jojo is the main character of JJBA, JoJo’s Bizarre Adventure).

I was deeply impressed, and it gave me the idea to create lineal figure images by re-organizing duplicated elements. So, if I have a simple element, and I have a image to show me where should I put those elements, I should be able to code a human’s portrait by mapping the elements onto the canvas according to a portrait image! Here’s how I started:

 

Preparation of Portrait Image

In order to provide an image that is easy for P5 (and me) to interpret the locations of each elements, I need to create something like a black and white version of the portrait image – the black color indicating something should be drawn on this pixel, and the color indicating nothing should be drawn. Here’s how I turned a portrait of me (Thanks Kiki!) into this “duo-color drawing map” using image thresholding.

 

Draw the Element

To make this a “self” portrait, I looked for something from myself – the abbreviation of my first name, YG – and turned it into a visual element using P5 primitive shapes.

 

Load the image into P5 and Set Up Units

Now I have the image map and the basic element, I can put them together! With the loadImage() API, I loaded the image into P5, and used loadPixels() to get the pixel array of this image. What’s left is simply to read each pixel to see if it is black and an element should be drawn at the same position on a new canvas, or it is white, meaning nothing needs to be drawn. Another thing is that on the new canvas, a pixel is too small to hold my element. So I need to scale it up a bit by using a “unit” setting, which means instead of squeezing the element into 1 pixel, I’m now drawing it onto a “unit * unit” box, and treat each box as an pixel from the source image.

 

Final Work and Reflections..

After some final tweaks on the stroke weight and color, this is how my self portrait looks like. I know’s it’s still far from Hirohiko’s Jojo Wall, but given the limited time frame, I’m happy with the result. And most importantly, the creation process is fun! More improvements can be done regarding the choice of element (the choice of Hirohiko is the “Do” character in Japanese, which is sort of triangular and good for making up images), the arrangement of elements (taking advantage of the linear property and orientation of the “Do” character to create image), and of course, the visual design of the entire art work.

I believe AI is also be capable of turning linear images into similar styles. I’ll explore more on this in the upcoming days at ITP.

Visual Language – Week 1: Design Analysis

This week’s work is to choose a design I like and analyze its adherence to the principles of design. My choice is the Grandmaster movie poster.

 

Introduction

This movie is directed and written by Wong Kar-wai, a Hong Kong filmmaker “internationally renowned as an auteur for his visually unique, highly stylized work”[1], and also, his fragmented narratives and relentless delays. Came out in 2013, the Grandmaster movie was nominated for Best Cinematography and Best Costume Design at the 86th Academy Awards, and it was the winner of Best Film of the 33rd Hong Kong Film Awards.

In 130 minutes, the movie depicts the life of the Wing Chun grandmaster – Ip Man. Rather than an action-heavy kung fu production, this film is more like a narrative poem that delineates the relationships between Ip and the people around him. In the center of these relationships is the one with Gong Er, the daughter of a martial arts grandmaster from northern China – they’re friends, rivals, teacher & student, and perhaps, lovers for a brief moment. In a sense, it’s complicate. And this is what the poster effectively communicates.

 

Underlying System

The poster is divided into three sections: the title, the main art, and the bottom footer with names of actors, actresses, producers, etc. The proportion of the main art is approximately two times of the title and footer, which gives a stress on the intermingling, complex relationship between Ip and Gong. It’s worth noting that the main art seems to be balanced both vertically and horizontally – vertically, there are two lines of text, one at the top, the other at the bottom; horizontally, the two figures are standing face-to-face in symmetric positions in front of a symmetric gate, while the weight of the text at the top left is somewhat counter-balanced by the trees in the back, located at the upper right corner of the main art.

 

Hierarchy

The main art is eye-catching and comes as the most important thing in the poster by being the biggest and one-and-only bright element among the entire dark background (which works sort of like casting a spotlight onto a fight scene on a theater stage). The title section with large fonts comes next, while the footer text section remains small and low-key at the bottom.

 

Typefaces

—-  Top —-

 

—- Middle —-

—- Bottom —-

It seems that three typefaces are used: the Elan family, the Poster Sans Bold and the Bee One. The Elan family and the Poster Sans Bold are the two major fonts in the design, while the Bee One is complimentary and used for the less important description texts at the bottom.

 

Color

This is a design relying solely on gray scale – five major shades are used. The black color sets the basic tone of the poster and it gives an outline to the primary figures; the dark gray depicts facial details of the two characters; the medium gray is used for title fonts; the lighter gray lays a serene backdrop (e.g. the looming mountains and the trees) to contrast with the rattling fight scene; and finally, white as the source of light.

I think this gray-scale setup rhymes with the overall tonality of the movie: Ip’s life-long struggle and fight for recognition and identity, and Gong’s tragic fate (who was injured terribly in another fight, then turned to opium, and died shortly after). In an age of political turmoil and drastic societal changes, the destiny of even the most powerful martial artists remained highly fragile. Looking back, the history was heavy and dark; Ip and Gong, nonetheless, kept fighting for their bright ideals – that’s my interpretation of this visual design.


[1] https://en.wikipedia.org/wiki/Wong_Kar-wai