Projection Mapping on 3D Surfaces – Workshop / Spaces 2014 – What we do and who we are?

View Github Repository Open presentation in a new window

schprc

See all presentation from schprc

Projection Mapping on 3D Surfaces – Workshop / Spaces 2014 – What we do and who we are?

0 0

spaces-mapping-workshop-october-2014-slides

Projection Mapping Workshop for the Spaces res

On Github schprc / spaces-mapping-workshop-october-2014-slides

Projection Mapping on 3D Surfaces

Workshop / Spaces 2014

by Irina Spicaka and Krisjanis Rijnieks

What we do and who we are?

For the past years we have been working with different projects that usually involve

  • Graphic design
  • Programming
  • Projection mapping
  • Electronics

.. as well as creating art installations and

  • VJ-ing
  • Curating
  • Organizing workshops
  • And audiovisual events

Irina Spicaka

Irina holds BA and MA in Visual Communication and Audiovisual Arts, has studied at the Art Academy of Latvia and Brera Academy in Milan, Italy. In her MA thesis, she was focusing on projection mapping, generative art and audiovisual composition.

Irina works as a graphic designer, curator as well as journalist in fields related to art, business and technology.

Krisjanis Rijnieks

For the past few years, Krisjanis has been exploring different media, programming languages and uses of technology and finally has arrived to a point where he can see how to finally complete his MA studies.

He is a student at the New Media MA programme of the Aalto University of Art, Design and Architecture in Helsinki and also has a background in painting.

CodeBark and CC4AV

We run a small digital media studio CodeBark and are also developing the CC4AV (Creative Coding for Live Audio and Visuals) platform that works as a placeholder for workshops and other activities that we have done in Helsinki (Pixelache pfatform), Riga (Skaņu mežs festival) and Helsingor (Click festival) so far.

We are also collaborating with

Our Projects

Latest projects that have been done by us in the past years.

Catface

Projection mapping and generative, real-time animation project.

Prism

Projection mapping and generative, real-time animation project.

Metasphere

Projection mapping installation and audio-visual performance.

Elves on Plane

Projection mapping on one of airBaltic airline planes for their annual Christmas campaingn. In collaboration with Cube digital agency.

You and Now

Interactive projection mapping installation within the annual Glow Riga festival in Riga, Latvia, November 2013. In collaboration with Mooz agency

Process

Audiovisual media installation, consisting of generative audiovisual content and projection mapping.

Subatomic Particles

Audiovisual event during the White Night 2013 in Riga, Latvia.

CC4AV Event Series

Creative Coding for Live Audio and Visuals event series with workshops and audiovisual performances.

School of Machines 2014

We did Projection Mapping on Spatial Objects workshop within this year's School of Machines in Berlin, Germany.

Click Festival 2014

We did Projection Mapping with the Raspberry Pi workshop within this year's Click festival in Helsingor, Denmark. In collaboration with Pixelache Helsinki.

More

Projection Mapping

Multiple Meaning of Projection Mapping

The term mapping highlights the role of space in the context of visual art as this term includes a function – real time projections adjusted to spatial objects.

The term mapping is also used in the computer sphere, for example, aligning peripherals with software parameters. It is possible to write on a personal computer’s screen due to the mapping of the keyboard with software functionality.

Projection mapping or real time projections adjusted to spatial objects is a recently created set of electronic audio visual media methods which allows to surpass standard-size projections multiplying the picture, as well as to adjust the picture for different surfaces.

A brief history

While projection mapping has recently exploded into the consciousness of artists and advertisers everywhere, the history of projection mapping dates back longer than you may imagine. If you try Googling for “Projection Mapping” you won’t find anything older than 3 years. That is because projection mapping’s older, academic name is “Spatial Augmented Reality”. The field is also known as “video mapping”, but projection mapping seems to be winning out in the United States. For the purposes of this history, I’m only including work that considered projection onto an arbitrarily complex surfaces. Projection onto flat and cylindrical/spherical surfaces has a much older history and goes back to the invention of cinema. Read more in an article with illustrations The Illustrated History of Projection Mapping by Brett Jones.

Inventions chronologically

The first known instance of projection onto a non-flat surface dates back to the 1969 opening of the Haunted Mansion ride in Disneyland. These were accomplished by filming head-shots of the singers (with 16 mm film) and then projecting this film onto busts of their faces. The next projection mapping instance comes in 1980, with the immersive film installation Displacements by Michael Naimark. In this art installation a living room with two performers were filmed with a rotating camera, then the camera was replaced with a projector. The result is rotating projection mapping. Disney not only pioneered the technology of projection mapping, they also made the earliest patent in the space in 1991. Entitled “Apparatus and method for projection upon a three-dimensional object“. It essentially describes a system for digitally painting an image onto “a countoured, three-dimensional object.”
In 1994, GE also has an early patent for a “A system and method for precisely superimposing images of computer models in three-dimensional space to a corresponding physical object in physical space.” 1998. Projection mapping really started to get traction when it was pursued in academia. “Spatial Augmented Reality” was born out of the work by at UNC Chapel Hill by Ramesh Raskar, Greg Welch, Henry Fuchs and Deepak Bandyopadhyay. It all got started with a paper The Office of the Future. The Office of the Future envisioned a world where projectors could cover any surface. Instead of staring at a small computer monitor, we would be able to experience augmented reality right from our desk. This means we could Skype with life-size versions of our office mates, view life-size virtual 3D models. This work even featured an early real-time, imperceptible 3D scanner (like the Kinect).
1999. You may know John Underkoffler as the designer who invented the Minority Report interface, and the Chief Scientist of Oblong Industries, Inc. But before that, he pioneered some of the early work in interactive projection mapping. He introduced the concept of the I/O Bulb (Input/Output Bulb), namely a projector coupled with a camera that could one be as ubiqitous as a traditional light bulb. In 2001 Raskar continued with interactive projections and paintings on still and moving objects. in 2004. From there Raskar went on to explore moveable projectors (predicting the pico projectors of the future). These hand-held smart-projectors are aware of their position and orientation through a variety of sensors. They demonstrated using smart projectors to aid in warehouse inventory and maintenance. In period of 2005 to 2006 Oliver Bimber explored projecting onto paintings and converting drapes into projection screens. See more with visual examples in article The Illustrated History of Projection Mapping by Brett Jones.

Why It Has Been Invented?

The reason why the projection mapping technique (as we know it today) was invented and developed further was a problem when there was no place for a screen at an event venue. The projector could also be put in a place somewhere in the room where the beam of light breaks partially on the ceiling, partially on one wall and plus on some other surface, thus being absolutely useless. There was no appropriate software at that time yet and specialists of the field worked out – if gamers had various advanced possibilities for using a room in 3D environment then VJs could use it dividing the room by walls and thus providing a more in-depth overview – getting beyond the classic rectangular screen. Read more in an article Spatial Projections by Irina Spicaka. Starting from 2011 commercial companies have started to develop the projection mapping as a serious, progressive technology. Nowadays if you are skilled enough you can make your own app and use it in your work and share it with others.

How can it be categorized

Fixed Surface Projection MappingProjection mapping on buildings or objects that are unlikely to change or, so to say, carved in stone. This allows much more predictable workflow for content creation. Variable Surface Projection MappingProjection mapping on previously unknown surfaces. This limits the content creation to elements of subjective choice. One might prepare videos for different types of surfaces (quads, triangles, circles) or generative visuals with adjustable parameters to fit many scenarios. Custom Surface Projection MappingThe type of projection mapping that we will do in this workshop. It allows the most flexibility as you are the architect of your own surface and the author of your own visuals.
Back to the first type it should be noted that when making projections for certain surfaces, the work is first done in 3D and animation software — classical animation methods are applied, such as lightning and special effects, narrative. The most classic example is projection mapping on architectural facades used at light festivals. The buildings are modeled first (or ready-made, modeled prototypes of them are used) and the story is created taking into consideration the design of the particular building. The process is connected with very accurate work where each mistake (let it be no more than a millimeter) both, in animation process or in the projection on the particular surface can be seen as a bias which looks like a defect. Therefore this process cannot be associated with real time performance. Read more in an article Spatial Projections by Irina Spicaka.

Where projection mapping can be used?

Tools

Decent software for creating media content and projection mapping.

Sketching tools

Adobe Illustrator Commercial Inkscape Free and open source Adobe Photoshop Commercial Gimp Free and open source

3D Modeling tools

Blender Free and open source Cinema 4D Commercial Max3D Commercial Maya Commercial

Projection Mapping

MadMapper Commercial Model8 and Resolume Commercial TouchDesigner Commercial LPMT Free and open-source Keystone Free and open-source VPT7 Free

Visual programming languages

Quartz Composer Free Max/MSP Jitter Commercial Vuo Commercial TouchDesigner Commercial

Text based programming languages

Processing Free and open-source openFrameworks Free and open-source Cinder Free and open-source

Game Engines

Unity 3D Commercial Unreal Game Engine Commercial Blender Game Engine Free and open-source

Frame-based animation

Adobe After Effects Commercial Open Shot Video Editor Free and open-source Pencil Free and open-source

Protocols

Software that can bridge visual output from one software to another in real-time.

Syphon Free, open-source and Mac only Spout Free, Windows only

Tips and Tricks

Best practices from experience.

1 Test it first

Before becomming too artistic, test if the technology works as intended.

2 Backup working versions

If your solution works, save it somewhere safe so when you continue to add changes, you can always switch back to the version that actually did work.

3 Know your surface

The more precise you will know the projection surface, the better. Especially when mapping outdoors on buildings.

4 Sketching helps

Make sketches and storyboards before doing it for real. Going through the process that way initially will help you overcome difficulties before you encounter them in the production process.

5 Always have your own adaptors

And cables. It might save you as well as others. Always have an extra cable and adaptor. It can make you the hero of the day. Or night.

6 No screensavers

Switch off screensavers and sleep for both, power and battery modes, on your laptop.

7 Cold corners

Disable hot corners or Exposé as it ruins the magic when a spectator notices what operating system are you using or that you have the same mess on the desktop as you have.

8 No instant messaging

Disable Skype or any other kind of messenger or service that tends to irritate you when least necessary.

Introduction to Blender

Creating a 3D surface for projection mapping in Blender.

Requirements

  • A computer
  • A 3-button mouse
  • Keyboard with numeric keypad (optional)

Tasks

  • Learn Blender basics
  • Create your own 3D object
  • Export it for assembly in real life

Install Blender

Download Blender from blender.org/downloads Install it on your computer Open it

Briefly about Blender

  • Blender is a free and open source 3D modelling suite. It supports modeling, rigging, animation, simulation, rendering, compositing and motion tracking, even video editing and game creation. Advanced users can code in Python to customize the application.
  • The first version of Blender has been released in the summer of 2000
  • Blender is cross-platform and runs equally well on Linux, Windows and Mac computers

3D Computer Graphics

  • Three-dimensional representation of coordinate values in 3D space
  • Based on Cartesian coordinate system usually
  • Uses a process called rendering to represent 3D objects on a 2D screen
Source: Cartesian coordinate system (Wikipedia)

3D Objects in 3D Software

  • Consists of vertices
  • Vertices form faces
  • Faces form meshes
  • Meshes form objects

Vertices

  • Vertices are Cartesian coordinates of a point in space
  • Each vertex has a X, Y and Z coordinate
  • (1.2, -4, 2.5) - this is a vertex as the computer sees it

Faces

  • Are supposed to be flat surfaces most of the time
  • A face is formed by no less than 3 vertices
  • The simplest form of a face is a triangle

Meshes

  • A mesh consists of one or more faces
  • It can be a open or closed 3D object
  • One or more meshes form objects in Blender

Normals

  • Each face has a normal
  • A vector that is usually perpendicular to the face and pointing outwards the object
  • Is used by the rendering engine to calculate lightning

Blender Shortcut and Cheatsheet

A couple of images that have been made to make the beginning of your Blender modeling life easier.

More about Blender

A couple of links to websites that can help you get confortable with Blender

  • blender.orgBlender main site tutorials. All the basics are there.
  • blendtuts.comAll kinds of Blender tutorials from the Blender user community.
  • YouTubeA lot of questions about Blender can be answered by searching in YouTube.

Exporting for assembly

To make a 3D object out of a 3D model we have to export it as a 2D surface

Install Export Paper Model plugin

Download Export Paper Model plugin for Blender:github.com/addam/Export-Paper-Model-from-Blender Open Blender User Preferences by hitting Ctrl + Alt + U while in Blender Go to Addons tab Click on Install from File button and select the downloaded io_export_paper_model.py file Enable the plugin by checking the box next to the plugin name in the Addons section

Prepare the Model for Exporting

Make sure that you use the correct units (metric) in Scene properties Make sure that the object has scale 1 along all axes If not, go to Object › Apply › Scale to make the current size of the object to be the default scale of it Triangulate all faces by entering edit mode (hit Tab), selecting all faces (hit A) and going to Mesh › Faces › Triangulate Faces or hitting Ctrl + T

Export Paper Model

Select the object Go to File › Export › Paper Model (.svg) Adjust properties (detailed explanation in next slide) Chose destination folder and file name Hit the Export Paper Model button

Export Paper Model Properties

  • ScaleFor precise output use scale 1:1 by entering 1 in the field. If you will attempt to export a cube that is 20cm wide, high and deep, also the exported faces will be 20cm wide and high.
  • Page SizePage size that will be used for laying out the unwrapped faces of the model. It should be bigger than the final drawing or the output will be scaled down to fit the page.
  • Unfolder DPIDots Per Inch or resolution that will be used by the plugin to generate the drawing. Set it to 72 as it is the default screen resolution.
  • Other propertiesYou can skip other properties as they depend on subjective taste. Experiment with them and adjust if needed.

Projection Mapping with PiMapper(and MadMapper)

We have come so far

Summary

We are going to use our computers and available projectors to projection map a surface that has been prepared for this workshop.

A surface?

The surface for projection mapping has it's story on it's own. It has been created in Blender and then made real with the help of a lasercutter.

How is this going to work?

Split into groups. Download and install software. Create image or animation source. Have fun with projection mapping!

Software

We are going to use PiMapper during this workshop. And if you like we can go through MadMapper as well (only Mac OS X users). Demo version is free, but you won't be able to save your projects with it.

PiMapper

How to use it and why.

Summary

PiMapper is being developed by Krisjanis Rijnieks as the openFrameworks projection mapping addon mainly to use in projects with the Raspberry Pi devices. It is cheaper way to create standalone mapping installations or portable instruments for audiovisual performances. Imagine yourself making 10 generative mapping pieces for an exhibiton which is supposed to stay at the gallery for 8 weeks. Would you use 10 Mac computers? Don't do it, it is too expensive!

It's easy to use it!

PiMapper has 4 modes:"1" (on keyboard) - Presentation mode "2" - Texture mapping mode "3" - Surface editing mode "4" - Source assignment mode

Presentation mode: This mode is activated once the application starts up. It does not show anything else except the final projection mapping as it was saved previously.Texture mapping mode: In this mode you can adjust the texture coordinates of the surface you have selected in the surface editing mode.

Surface editing mode: Here you can select, move and distort the surfaces you have created.Souce assignment mode: After you select a surface in surface editing mode, activate this mode to be able to choose a source for the surface. Afterwards you might want to go to the texture mapping mode to adjust texture coordinates.

Other shortcuts:"i" (key) - Show info (function)"n" - Add triangle surface"q" - Add quad surface"r" - Add random triangle surface"f" - Toggle fullscreen"s" - Save composition"BACKSPACE" - Delete surface

MadMapper Cheatsheet

A couple of images that have been made to make the beginning of your mad mapping life easier.