Professional Documents
Culture Documents
1. Introduction
The motivation behind doing this project
was that we wanted to work on something
fun. We thought about several dierent
ideas before settling on making an
application that creates ASCII Art. The iOS
applications first step is to take 640 480
video from the back camera. The application
then takes the raw video frame from iOS
AVFoundation (as uint8_t *, BGRA format),
converts it to grayscale through iPhones
ARM64 neon instructions; then the
grayscaled frames are passed to a pre-learnt
decision tree to determine which
corresponding ASCII character will represent
the section of the frame; we finally display
the output string of characters on the
iPhones screen. The decision tree that we
are using is the trained decision tree
provided by the paper author. In their
implementation, they encoded each ASCII
character as rendered grayscale pixels. This
yields a better performance in terms of
displaying the pixels on screen, but it limits
2.5 Coloring
In our naive implementation, we used the
first pixels color value in a block to color the
corresponding ASCII character (we also
increased the saturation of the color value,
so the color will be more obvious). Evidently,
using such approach will not result in a very
accurate color representation, and one can
easily come up with other color schemes
such as using average color in a block, or
using the dominant color in a block.
However, due to time limitation and the
complexity of computing such value, we left
out this particular feature. Adding average
color and/or dominant color text coloring
could be an improvement in the future.
2.6 iOS Framework
The majority of the core algorithms and
techniques that we used to transfer the raw
pixel input of the iPhone camera were
summarized above. However, it is still worth
mentioning the framework we used in the
iOS environment particularly to implement
this App. For camera input, we used
AVFoundation framework as it provides us
with raw input values; for displaying the
resulting image, we rendered the pixel value
directly onto a CALayer as its lighter than
the traditional UIView; finally, for text
rendering we used low level CoreText
framework and our customized subclass of
CALayer to achieve the maximum
performance.
Figure 1: Screenshot of our app running on four dierent modes. From left to right they
are: Black and White Rasterized, Colored Rasterized, Black and White Text, and Colored
Text.
3. Results
As demonstrated in our presentation, our
App ASCPI, running on a iPhone 6, could
transform and display the ASCII
representation of the iPhone camera input
in real time. 3 out of 4 of our display modes,
pixel black and white, pixel color, and text
black and white could achieve satisfactory
frame rate. However, since we only used the
color value of the first pixel from each block,
the colored version is not very accurate, and
this feature will be improved in the future
implementation.
4. References
Marku, N., Fratarcangeli, M., Pandi, I., &
Ahlberg, J. (2015). Fast Rendering of
Image Mosaics and ASCII Art. Computer
Graphics Forum(34), 251-261.
Khvedchenya, E. (2011, February 7). A very
fast BGRA to Grayscale conversion on
Iphone. Retrieved December 21, 2015,
from http://computer-vision-talks.com/
2011-02-08-a-very-fast-bgra-tograyscale-conversion-on-iphone/
(We also used Xcodes builtin
documentation)
5. Extra
Language: Objective-C
Platform: iPhone 6 (because of the
assembly code we used, it will only work
on arm64 architecture)
Lines of code: 800+
Team member and contribution:
Zhilin came up with the initial idea of phototo-ASCII conversion, found the Marku et al.
paper and their partial source C code that
our project is based on, reversed engineered
the trained decision tree provided in the
code, designed the live demo part of the inclass presentation and collaborated in the
design of the project website. Zachary
contributed to the initial dissection of the
papers code via some comments, the first