Augmented Reality: Becoming a Reality

The days of reading a bedtime story with a few pictures on the page is forever in the past. The digital version of books, billboards, and life as we know it will soon be using augmented reality to deliver content. We’ve been doing similar to this for a few decades by hiding easter eggs of content in our traditional media, the secret spot in the original Mario Bros. where you get the flute to change levels or an extra scene in a movie after the credits. Augmented Reality or AR, has been traditionally used to create content much like this to create secret treats that only a handfull of people know about. These modern easter eggs have been steadily picking up steam as innovative developers and savvy brands embrace the possibilities that this new medium offers. What was once a treat enjoyed only by a few technology lovers is now becoming something people look for in well done campaigns and media. Of course just like everything else, doing it is easy but doing it right is the real trick.

 

This week at SXSW Marvel Comics announced a comic book that will merge the digital and paper world of comics for the first time. Each comic will have an “AR” logo such as the one in the image above. These new comics will let readers see Iron Man fly off the page and take down the Hulk by simply downloading the free Aurasma app. Merging books and AR isn’t a completely new idea but offering it on such a large scale hints at the growing trend and appeal of the technology. Marvel certainly won’t be the last to adapt this tech, especially in the realm of children’s books. It won’t be long before we see Dr. Seuss’ books with AR worlds popping out of the pages.

 

The Marketing industry has been using AR for some time now as well, especially in the toy industry. However companies such as Domino’s, Starbucks and Volkswagen are joining in. For instance look at the Domino’s billboard above, they are now using AR to connect to customers and allowing them to download the app to make an order, see the menu, and find locations. All of these options are hidden on the billboard but can only be accessed with the AR application.

 

Take a look at Volkswagon’s marketing campaign for the new Beetle. They went all out and people responded in kind by downloading the app and getting rewarded with an AR treat that goes well beyond what the user was expecting. This sort of visual and tactile marketing to people through the use of their mobile devices creates a much more lasting connection to the brands that they engage with, the viewer isn’t being forced to look at advertising because it’s rammed in their face, they actively engage with it and initiate the conversation.

With brands jumping into the augmented world left and right, developments such as Googles rumored release of a pair of AR glasses has us all on the edge of our seats. What infinite possibilities await us when the entire world can be augmented without constraint? Will brands and Marketing campaigns respect the same user initiated paradigm when they can broadcast advertisements onto any surface or space? Will we be able to “un-augment” our world at will? I guess we’ll just have to find out.

AGENT X Digital+Live Awesomeness Awards w5

The fresh list of Digital+Live stuff from this week includes casino openings, hollywood scale models and the entire history of the universe. Give it a look and drop us a line.

 

For the grand opening of the Holland Casino Scheveningen a full scale interactive building projection that included robotic arms and infrared views of the casino’s interior got the crowd pumped up before the ceremony. It also allowed the first ball to be thrown into a projected roulette wheel while the crowd watched. Check it out…

 

Unit9 premiered this little gem at FITC Amsterdam 2012. Created using Unity and the Adobe AIR platform in Flash the game allows up to 3 people to play at once using their smart phones or tablets as controllers. Defending the city from an alien invasion while attempting to keep your errant frisbees from ramping up civilian loses sounds like a ton of fun to me. This is another great example of HTML5 Websocket technology being put to great use in public settings and it should be on sale soon in the App Store just in time for your next BBQ. Check it out…

 

Ever wonder what the history of the universe might look like if it was laid out on a timeline? Neither have I, but now that I’ve seen it I’m pretty fascinated by this tool. ChronoZoom is an open source community collaboration from Berkley, Microsoft Research and Moscow State University that allows users to visually navigate through the last 13.7 billion years. You might be humbled by exactly how small the existence of humankind is in relation to everything else that has happened. Check it out…

 

Now on to something a bit less serious but potentially just as impressive. The 50 foot wide scale model of Hogwarts castle that was used to film all of the movie series exterior shots is a piece of art and now you can take a look at it for the first time outside of the silver screen.  The model has 2,500 fiber optic lights, hinged doors and it’s own lighting system. No wonder it took 86 artist roughly 74 years of combined human life to build it. Check it out…

AGENT X Digital+Live Awesomeness Awards w4

Alright, we’ve got a pretty big line up here. Everything from mapping out concentration levels in the minds of children, to capturing nature hikes in step-by-step 360 degree technology. With all of this new technology emerging its still clear that the real gems don’t just use the new tech, they use it for something that no one else thought of. Take a look at some of the best new projects you’ve never heard of before now. If you’re looking for someone to help you pull off something like this please feel free to drop us a line.

 

Varier asked B-Reel to help them out with creating designs for their new Variable Balans children’s desk line. Since the product was designed specifically to help kids concentrate B-Reel decided to see exactly what that might look like. They invited three children to help design the chairs by mapping out their brain activity and concentration levels and then using the resulting output as a basis for their designs. I’d hate to think what the fourth design would have looked like if they invited me to try it out. Check it out…

 

Wonderwall Studio is doing something both green and great at the same time. I think this is a much better route then a gaudy primary color as an accent wall for your office, but hey you don’t have to take it from me. Combine the unique look with the fact that even someone as clumsy as me could put this up with a nail gun and a level and I think the product pretty much sells itself. Check it out…

 

Ponk combines the best of old school classic gaming and new technology by grafting Pong and Kinect. If you’re looking to rent this thing or partner up they seem to be open to that idea as well. The videos of people demonstrating the Ponk table look pretty fun, I just hope the top of that thing is spill proof. Check it out…

 

City Peaks is a microsite created by London based agency Digit. In order to encourage their own employees to be more active and use the stairs instead of the lift they installed RFID card readers on the bottom and top of the building. The resulting microsite tracks each persons progress as they race to reach to top of famous London skyscrapers. This is a pretty interesting behavior modification program with it’s own set of motivations, sure Nike Fuel and Fitocracy boast huge social networks with loose social pressure but creating the same system in a microcosm certainly seems effective as well. Check out the project and then check on their progress…

 

Nature Valley Trail View is the child of Google Street View technology and National Park Splendor. Instead of imagining what it might be like to hike through the Grand Canyon you can instead step through the 360 degree views of the trail second hand. Which is great for guys like me who are terrified of tripping and falling into gorges. If the Grand Canyon isn’t really your cup of tea you can try out Yellowstone or the Great Smokies.  Check it out…

AGENT X Digital+Live Awesomeness Awards w3

Sorry for the delay in this weeks post, an unfortunate family-wide bout of the Norovirus pushed getting this up for a few days. Regardless of the delay hopefully there’s something in here for you to grab some inspiration from, it’s all pretty awesome as the title implies.

 

The University of Oregon’s new Ford Alumni Center was built by a group called Second Story. It’s one of the coolest media installations I’ve ever seen. If anyone out there is looking for a great example of how to do something like this the right way, this is it. It even has it’s own CMS system that allows the alumni center workers the ability to update and edit content themselves, I’ve got a feeling enrollment might be on the uptick for these guys. Check it out…

 

After using this app for a week I’m still just as excited about it as I was when it was still in development. I really can’t speak highly enough of its UI, as it’s the perfect marriage of native interface and intuitive gestures. Sure it doesn’t have all of the over-complicated things like calendar integration or push notifications built into deadlines but the pure simplicity of the app lends itself toward a more flexible and useful tool. Also my wasteful use of post-it notes has declined by 100%, which would make this the first app I’ve ever used with a positive green impact. Check it out…

 

Alright, if you don’t have a soft spot for collectible plastic nonsense then you’ll have to excuse me for this section ahead of time. I think these are pretty clever and perfectly retro, but then again I’m a huge fan of the original Transformers, Star Wars, Comic Books and of course quick file transfers between co-workers. Check it out…

 

Bear 71 is a pretty unique mix of documentary, technology and imagination. After watching a short intro you get to explore the rest of the interactive map on your own as the site pieces together the issues created when mankind expands and the animal kingdom is forced to adapt. Check it out…

AGENT X Weekly Digital Awesomeness Awards v2

This week has gone by pretty quick but I managed to snatch a few gems out of the ether to share with everyone. From browser based flash games that highlight the rise of device interoperability, to analog versions of the fridge, theres a little something for everyone. Check it out and drop a comment at the bottom, cause you know you want to!

 

Jihyun Ryou is bringing back old school food storage. It never fails, every few weeks I go to grab vegetables out of the crisper and they’re nothing but mush, then I remember that I bought then half a month ago. Maybe something like this could help me out, at the very least it would cut down on my electric bill and make my house a bit more green.  Check it out…

 

Mammoot Control is a retro style flash arcade game built by German based UFOMAMMOOT which utilizes the new websocket protocol. The game allows you the option of using you’re iPhone as a game controller instead of a keyboard. Considering what we do in the realm of live events, something like this seems like a solid hit for a brand if utilized in the right way. Check it out…

 

When I was a kid I loved everything LEGO, but Max Shepard’s biomimetic(yeah don’t worry, I had to look that one up as well) robot arm made from 100% LEGO parts puts anything I ever made to shame. Of course LEGO didn’t make things like linear actuators when I was a kid. Check it out…

 

Matt gets the credit for passing this one along to me and even though I’m no musician this video got me pretty excited to try out this thing. It’s like a great mix of the new Tron movie with the puzzle based play-style of Myth and a synthesizer laced overtop of it.  This gem is being cranked out by a three man shop called Phosfiend Systems in Canada and they have a great dev blog themselves over at fractgame.com.

 

Showcase for Dutch artist Casper Faasse’s work with a flash / html hybrid site that allows for great SEO and multimedia content delivery.  I especially like the way he reinforces the physicality of your own computer screen by pressing the iPad up to it while displaying his work.  It’s subtle yet very clever, in a world full of sites that attempt to draw you into the screen he chooses instead to remind you of it’s existence.  Check it out…

AGENT X Weekly Digital Awesomeness Awards v1

There are plenty of things out in the wilderness of the internet that fly under the radar and bubble up at unexpected places. As part of the creative team at AGENT X I spend a part of each day making sure I stay abreast of as much cool new industry activity as I can without getting lost in any rabbit holes along the way. In my daily effort to find at least one thing out there that is both awesome and relevant I sometimes find things that surprise me, enjoy.

 

Filmmaker Jacob Sutton spent multiple days in the -25 degree temperatures of France capturing pro snowboarder William Hughes as he carved up the hills in a glowing white LED suit. The surreal quality of these shots create some incredibly gorgeous visuals and are hauntingly reminiscent of astronauts on the moon (although the soundtrack really solidifies that). Check it out…

 

B REEL has posted up a video highlighting their interactive multiplayer game installation called EELS. The whole project is a great showcase for what can be done using device interoperability, projection mapping and a lot of careful planning with a great eye toward usability and multi-person interactions. Check it out…

 

Pretty cool visualization of the movement patterns of Norwegians using tax return data from 2011. Written in C++, the code behind this would make your brain bleed but Even Westvang’s done a number of these projects. Check it out…

 

Even though these are actually all digital they look so close to being possible that it messes with your mind. This paper craft style is pretty original and unique, take a look at the larger scale projects Jeremy Kool is working on as well, including the interactive iPad application currently under development. Check it out…

 

Andrew is responsible for finding this and sending it on to me and I can’t thank him enough, since this might possibly be the best toy ever, except for the original G1 Transformer toys . Either way i’ll still buy two of these as soon as they’re ready if he puts in a flamethrower attachment. Check it out…

 

Alright, by this point I’m sure you can tell that I’m a sucker for cleverly done data visualization. Especially when it has great art direction, I especially love the submersible and the penguins in this HTML5 site. The Astronaut on the moon is pretty amazing as well. Check it out…

 

If this is the first you’ve seen of Wayne Dorrington’s Star Wars Iconoscopes I advise you to go through them in order if you have the time. Wayne does such an amazing job of summing up each scene that even though I haven’t watched the original movies in the better part of a decade I can clearly tell exactly whats going on.

Episode IV - A New Hope

Episode V – The Empire Strikes Back

Episode VI – Return of the Jedi

 

Project View // Alpha

Even with the very best project management system in place it can often be fairly difficult to help everyone in and outside the office understand where all of the various projects stand and who is working on what. Having something that very clearly provides that sort of oversite is incredibly beneficial to not only the project management team but also upper management and and all of the ‘doers’ on your team as well.

This has been a topic of lingering and consistant discussion at every agency I’ve worked at for so far in my career. It also happens to be something that I have yet to see implemented really well in a way that helps out every part of the team, not just a specific section. Sure, it’s easy to throw up a whiteboard and scribble out all of the projects your teams working on but you have to reorder things constantly and no ones checking that board on a regular basis but the PM. Or you could try using something like Basecamp that lays out all of the milestones and events your team has to get done, but I’ve found that using a calendar feed to help with your deliverables just muddies up your entire calendar.

Project View // Alpha is our attempt to create a piece of software that works for our team, which means it may not work for your team. The features below are things we wanted right away realizing that we’d need to add in an admin interface later.

  • Highly visual and intuitive layout
  • Ability to add individual employees and vendors as resources
  • Ability to add project deliverables and assign client, status, due dates and estimated hours per resource to it
  • Calendar style view of deliverables for the next 3 weeks
  • Ability for the software to generate and reorder deliverables based on a priority ranking system, more on this below
  • Ability to place deliverables on hold and still view them
  • Ability to easily see each resource and the estimated hours across all projects for the next 3 weeks along with an additional lump sum of the remaining hours beyond 3 weeks.
  • XML driven data structure for ease of use
  • Ability to view the output projected onto one of our office walls, through a browser page login off site or through a desktop app that pulls in the latest feed if an internet connection is available.

Now it just so happens that these kind of software applications are a passion of mine and so it didn’t take much for me to sit down one weekend and start prototyping something in Flash. Why Flash you ask? Well, there are a number of reasons really, and a few of them get back to the last feature in the list above. We really wanted this to be something that was viewable in and out of the office, with or without an internet connection and something that offered a bit of interaction. Animations, design and overall performance are great pluses as well with Flash.

In the long run after the final design is locked down we’ll be transitioning this over to an AIR application that we can distribute internally so everyone can see it at all times as well as a widget and an online portal. We’ll also be using a centrally stored XML feed on a server to download updates to the tool and keep everyone up to date as the PM staff adjusts things.

Starting off we realized that the most important part of this software was it’s ability to clearly show all of the current deliverables being worked on in the office and prioritize them for us. In order to do that the interface would need to be comprised of some sort of list that could expand downward from most to least urgent deliverable. Here is a look at our first pass for the list view of the project deliverables. You can see below each deliverable has the client / project information, resource allocation by initials, due date and calendar view.  The list element also displays a priority ranking that adjusts the deliverables ordering based on the number of resources allocated, hours left on the deliverable and due date for the project.

The system also has a resource overview element that visually totals up all of the hours per resource so our project managers can help load level the expected workloads. The element breaks out the projected hours per resource into the same week period displayed by the calendar view and adds up any additional hours for projects that go beyond the time frame.

Stay tuned for some more updates as we make progress and adjust the software to fit in better with our SCRUM meetings and Squash sheets.

Interactive Welcome Screen

// The Concept

We are starting something pretty exciting here at Agent X. In an attempt to make our office more connected, we are in the works of creating an interactive welcome screen that can also serve as an ambient control system for our entire space.

There is a lot that we want to do with this, and while we are still in the planning stage, we are sharing using this first post as a mind dump of components that we want to include.

// Components

In order to create an experience that is both engaging and useful, we don’t want to leave any possible element unturned.

  • Clock Resting state for the interface.
  • NewsNot that we are too busy, but for the occasional visitor waiting, it is a great way to pass time.
  • Virtual Check-in A visitor is typically looking for someone. With the virtual check-in feature, they can send an alert directly to the person’s computer notifying them of a visitor.
  • Attendance Possibly done using RFID, if en employee is not in the office, the system will show them as out.
  • Weather Our headquarters in Michigan, the weather changes constantly. From the time a visitor enters the office, until the time they leave, it may be completely different outside. Pulled from weather.com or a similar source, the weather feature is a fully-capable weather app with high definition visualization of current conditions and forecasts.
  • Company Information Elevator speech, portfolio elements, general company information, history timeline… you know, everything someone might want to know about our company.
  • Ambient ControlsThis is where I get really excited thinking about the opportunities. Nothing would say technologically adept than to have ambient controls such as lighting, temperature and music controls in a central location.
  • Agency Calendar For internal use mostly. A nice exercise in visualization of data, with the benefit of seeing company wide bandwidth. If possible, pulling in iCal information to show meetings and availability would be ideal.
  • The actual welcome screen We have tried this in the past by uploading the logo of a visiting client on the TV in the conference room. It was rather successful, but a bit time consuming, and requests to have display screens typically came last minute. Nonetheless, with proper planning this would be a nice touch to let potential clients feel even more welcome.

// The Build

Some basic ideation and planning has been done as to how the finished frame will look. We want to create a unit that has the ability to not only house and feature the touchscreen, but also hide the other various components that enhance the overall experience, i.e. Kinect, speakers, RFID reader…

// Technology

I’m sure we are going to find different / more elements that we are going to need  in order to create the best experience possible.

/ Hardware

  • mac mini The power behind the experience. Compact enough to hide.
  • 42″ Touchscreen monitor Sufficient size we think. While we want to make this an immersive experience, we certainly want to avoid the overwhelming
  • Microsoft Kinect Detect voice and movement.
  • Speakers Part of creating ambience in the office will rely on sounds and music.
  • RFID detection

/ User Interaction

  • Touch Ahh, the physical touch. Most of us are accustomed to it through smart phones and tablet; the challenge will be to create and experience that would rather be had on a screen that would be better than not…
  • Movement For us, this is key. Most interactive displays detect touch, but we want to this experience to go further. When you walk up to the screen, or walk past it, it recognizes a potential user. This creates a much more organic experience between the user and the interface.
  • Voice Also done with the Kinect.
** None of the examples in the main image are the work of Agent X, but instead of the incredibly talented creative individuals that continue to create and inspire. **

Kinect – OpenNI and Processing

(part 1 of series)

Why OpenNI?

We’ve all heard of the Kinect device, or the Microsoft XBox 360 by now and those of us in the developer community are aware that Microsoft has released a second version of the Kinect SDK for Windows. When asked to try to uncover the capabilities and possibilities of this new technology the Microsoft SDK is where I began.

Note: A standalone Kinect is required as the units that come with the XBox 360 use a non-standard USB connector. An adapter is available but rather hard to track down, so I went with the extra camera available at most big box retailers.

While the demos are cute, and installing the camera and SDK is fairly simple (Windows 7 only), the Visual Studio development environment created a super steep learning curve.

Enter OpenNI and Simple OpenNI open source drivers and libraries that can be used whether you work on a Mac, PC or Linux and like to code by hand using Processing.

Depth Camera

The camera consists of (left to right) an infrared projector, infrared depth camera and RGB camera. In addition to the camera there are 4 microphones and a servo motor in the base. WALL-E anyone?

Above is what the camera sees with the depth camera at left and RGB camera at right. Not a whole lot to look at, but notice the shades of grey and black in the depth camera image (lighter shades the closer the subject). The infrared projector projects thousands of infrared dots that the depth camera uses to measure distance.

 

Point Cloud

The image above shows how I was able to create shapes within the point cloud that when touched will trigger drum sounds. Being a musician myself, the idea of a virtual drum set was too cool to pass up. What about the angle of the view? Yes, the kinect camera is right in front of me so how did I get that shot? One more thing to add… the image rotates in 3D.

3D

In the video clip below from daniel shiffman you can see the effect in action:

Another kinect sighting

While browsing the flickr gallery for the New Zealand band Pajama Club I came across this behind the scenes shot from their upcoming video:

Here’s a link to a preview of the video giving some clues as to how the director is using the kinect, OpenNI and Processing:

Click here (use Chrome, Safari or Firefox for best results)

What do you want to see created with this new technology? I’ll be writing another post soon to let you know where I’ve taken my ideas.

The Sphere

We’ve been working with this piece of technology for about a year now and the more we do with it the more possibilities we seem to unearth. In its most simplistic form the sphere is a pretty cool and novel piece of technology that lends itself well to group settings and events. It can be used for things as simple as projecting video in 360 degrees or as complex as fully interactive touch sensitive applications built to spec for clients. But where we’ve begun to really see some cool new possibilities is the area of mobile device interoperability.

This idea here is to build up an application for the sphere that can allow for multi-player cooperative game play using your mobile device as a controller and possibly even motion capture from a Kinect to create a engaging experience that will become the RnD framework for client pitches later.

We’ve also been looking into interesting ideas of combining AR (Augmented Reality) and the Sphere, heres a look at a screenshot grabbed out of one of our iPads running a prototype app to test if the curvature of the sphere would throw off the image recognition software, initial success seems promising in this area.

Here are some videos of the initial prototype apps we’ve created for brands like Symantec and Redbull to help illustrate soem use cases for the sphere specific to their own brands.

 

 

The two major challenges we have here are hardware and software. From the hardware perspective we already have a pretty solid framework to start from since were dealing with a mostly complete product and modifying it to fit our needs. The custom lens, projector equipment, LED touch ring and sound system have already been tested out and are ready to go. Aside from that the rest of the hardware will end up off site in the form of a server and wireless network setup.

The software component of this is where the real tricky parts are going to come into play. In the past we’ve used Unity 3D to facilitate prototype development for the sphere and right now we are planning to use it for this RnD project as well. From there were going to need to hook in Touch OSC and some multiplayer capability for the prototype.

Right now we’re focused on getting some open source components up and running on the device so we can test out the general functionality and get a better idea of the strengths and weaknesses of each software option.