An Unprecedented View of Earth & Space: 360-Degree Video

Well, here’s the big thing my friend Greg Pitner and I have been working on for RPI-SEDS the past few weeks! I am delighted to present an unprecedented approach to experiencing Earth and space. In this 360-degree immersive panoramic video, you will witness an abridged journey of our recent high-altitude balloon launch. If you want some background, read my tragically comical story about what it took to get to this point. Take a look:

UPDATES:

Here’s a list of some mentions we’ve received:

 

The Video

NOTE: For best results, let the video load for a bit before playing, to minimize choppiness.

 

The “Flattened” Version (watch in full quality!):

 

Why I’m Excited: “Grassroots Space”

Let’s face it: space isn’t sexy anymore. The zeal for the cosmos that once pervaded the American public consciousness has gone flat, and we’re not going to rediscover that passion through politicians or NASA administrators. It will reemerge through the efforts of entrepreneurs in the private space industry and through the legions of professionals and students inspired by their actions.

Three battlefronts define the growth of this new space movement. At the boundary of technological knowledge, space entrepreneurs fight for business models that can bring sustainable profits. In the political arena, commercial space advocates are creating the necessary relationships with NASA and the U.S. government so that the industry can grow.

The third battlefront is an intellectual mission to inspire the future generations of space scientists, entrepreneurs, and workers. Educational outreach efforts aimed at students primarily define activity on this front. The national student-run organization Students for the Exploration and Development of Space (SEDS) is one example, dedicated to fostering enthusiasm for space among young people. I founded the RPI chapter of SEDS to bring that excitement to my campus by engaging students with space-related projects.

Our first effort, the high-atmosphere balloon, is an innovative variation on this increasingly common project. The idea that regular students can realize grassroots space projects like this embodies new opportunities to inspire people unlike ever before. That is the central idea behind presenting the high-atmosphere in a 360-degree interactive medium.

 

How We Did It

A block of extruded polystrene (i.e. foam) served as an effective vessel for an ARPS (for tracking), three GoPro HD Hero cameras, associated components, and handwarmers (to keep the transmitter batteries warm). We attached this payload to a helium-filled balloon and a drag parachute.

We mounted the three cameras on a piece of plexiglass we screwed to the top of the foam block so that they looked outwards, radially equidistant at 120-degrees. Shooting at 720p mode offered a 127-degree field-of-view for each camera, leading to approximately 7-degrees of overlap between any two adjacent cameras (at best).

In retrospect, we needed more overlap between adjacent camera field-of-views. We initially intended for a seamless (i.e. no visible frame edges) 360-degree video, but we did not have sufficient overlap to achieve this realistically. Fortunately, I think the final product demonstrates the original vision effectively enough.

The Payload

 

After successfully launching, chasing, and retrieving the balloon, video post-processing became the game. This is the workflow that resulted in the video above (omitting nitty-gritty details):

  1. In Final Cut Pro 7 (FCP7), create a custom-size canvas 3840x720px and place all three camera feeds, making sure that all feeds are time-synced.
  2. Tons of processing, including: skewing segments of movie to align horizons, image correction, cutting down movie to two minutes, adding music.
  3. Export video to an .mp4 using H.264 codec, saving at an appropriate quality for display on the Internet.
  4. Use krpano Tools to create a 360-degree panoramic video.

Had we the opportunity to do seamless stitching between cameras, we would have followed the following workflow (omitting details, again):

  1. In FCP7, align all camera feeds on a custom-sized canvas the size of three adjacent feeds, making sure that all feeds are time-synced.
  2. Edit the video length to your desired outcome.
  3. Export ALL frames from the three feeds as high-resolution images.
  4. Use PTGui Pro to automate batch stitching process for ALL frames.
  5. Import panoramic images as a movie in Quicktime Pro, then save as a movie file.
  6. Use FCP7 to edit video to final outcome (e.g. doing color correction, adding music).
  7. Export video to an .mp4 using H.264 codec, saving at an appropriate quality for display on the Internet.
  8. Use krpano Tools to create a 360-degree panoramic video.

 

The Launch

Reaching Space

A frame from one of our videos about 90,000ft above Earth.

 

Saturday, January 28 was a tremendous day. We finally realized a project fifteen months in the making! When I met Greg that morning, I told him this was a day we would remember forever. Read on.

 

The People Behind the Project

Our project had many phases, including an intense round of fundraising. I would like to thank all my founding club members and those who were there to help kick off the project. Here are the names in alphabetical order: Orian Breaux ’12, Alan Carey ’13, Zachary Clapper ’14, Nic Gaudio ’13, Jon Huang ’12, Brian Jennings ’12, Anson Koch ’13, Jeff Mockelman ’14, Mack Ott ’14, Greg Pitner ’12, Matt Scorza ’14, Paul Sicard ’12, Aaron Valentine ’14.

 

Background: The Idea is Born

In the Fall 2010 semester, I founded a club at RPI called “RPI Students for the Exploration and Development of Space” (RPI-SEDS). This college chapter of a national student-run organization is meant to foster engineering experience through space-related projects. Given our lack of funding and brand new status, I wanted a relatively accessible first project. In a conversation with Prof. Matt Oehlschlaeger, I learned about a project where a few hobbyists launched a high-atmosphere balloon to capture video and atmospheric data.

I presented this project to RPI-SEDS in a meeting, sparking great excitement. We had our project, and design was underway. I wanted to foster engineering experience among the members, as well as create an atmosphere of teamwork and engagement. We broke the project down into subsystems and assigned various members based on interest. Ultimately, we developed an innovative idea of capturing 360-degree immersive panoramic video using three HD cameras. After watching the concept on the Internet, I was ignited by excitement for a result that could truly make an impact.  Once we had a clear idea about the components involved, we initiated a fundraising strategy aimed at the Mechanical, Aerospace, and Nuclear (MANE) Department, various companies, and our friends and family. We raised about $1000 after a month of effort. By the end of the Spring 2011 semester, we left for summer break with a complete design and some components, like the APRS.

RPI-SEDS did not make significant progress during the Fall 2011 semester due to a lack of meetings, but everything came together in January 2012 during winter break. Greg Pitner and I completed the construction in about a week of effort. Our eagerness to launch our project led to decisions that reduced the complexity of our design. We went from a tear-drop shaped payload to polystyrene rectangular box.

 

Launch Day

Unable to sleep due to excitement, I met Greg on the RPI campus at 6:30am. We set up our launch site on the Class of ’86 field, ready for Alan Carey’s arrival with a tank of compressed helium gas. We readied the payload, ensuring the ARPS worked properly, the batteries were secure, handwarmers activated, screws secure, and cameras properly positioned. By launch at 8:29am, we had present three local media people (CBS 6 Albany, YNN Albany, Troy Record), Greg’s parents, a bunch of RPI students gathered around.

We released the balloon at the end of an audience “5-4-3-2-1!” and clapped as the balloon rose toward its inevitable burst at 90,000ft above the Earth. As we cleaned up, Greg and I had short interviews with the local news cameramen. I was tired, cold, and anxious to leave, so my responses were not very fluid. But hey, at least I made television at least three times that day! Here are some local media pieces:

 

The Pursuit: Take 1

And it began. My car became a search-and-rescue vehicle doomed to a fate far beyond what I predicted that morning. Greg and I, joined by fellow RPI-SEDS members Mack Ott, Paul Sicard, and Elisabeth Smith, faithfully watched my phone for APRS updates using http://aprs.fi (callsign KD2ABA) on the balloon’s position as I drove toward Vermont. Soon after, our friend and RPI-SEDS member Brian Jennings and his brother Chris embarked from his house in Merrimack, NH in pursuit. Excitement and fear produced some nervous conversation about whether the balloon would fail or land in the Atlantic. Updates every 30 seconds gave us the balloon’s speed, heading, and altitude. At 9:16am, the balloon reached its maximum speed at 246mph at an altitude of 29,039ft and heading of 74 degrees. Here’s another maximum: at 11:00am, the balloon reached a peak altitude of 89,777ft near the Lake Winnepesauke, New Hampshire region, coincidentally where I had predicted the payload would land. But let’s not get too ahead here.

At about 11:30am, trouble struck our search team as my car broke down without warning on I-91 at mile marker 44 near Springfield, VT. In a bizarre incident, my engine started sputtering until it failed to work. Coincidentally, the balloon landed within five minutes of the car breakdown, adding to the stress of the moment. The last signal we received indicated an altitude of about 500ft and a location DANGEROUSLY close to a river. We feared that the payload had landed in the river. Brian and Chris would soon find out as they ventured to the forest near Steep Falls, Maine for the recovery.

UPDATE: It turns out that my fuel injection system failed.

While stuck, I arranged for a towing truck to come and bring my car back to the Troy, NY region. At the same time, Greg called his parents, who were willing to drive from Troy to help us complete the journey. Greg’s parents and the towing truck didn’t arrive until around 2:20pm, so we found ourselves playing with our phones to pass the time. We even ordered a pizza for delivery, a fun moment that brought out a high-school senior delivery boy who actually applied to RPI for undergraduate. As Greg said, “That kid must think RPI people are the coolest ever.”

 

Greg Eating Pizza Along I-91N

Success?

Greg’s parents arrived at about the same time that Brian and Chris found the payload in Maine. We rejoiced as Brian texted me a photo of our equipment hanging from a low-lying tree. Of all the places in New England—mountainous summits, lakes, enormous trees—we were fortunate to retrieve the payload so easily. We all made new plans: everyone go to Brian’s house in Merrimack to exchange data and celebrate.

After my car was towed away, our team, now driven by Greg’s parents, sought out Merrimack. All went well until more trouble came our way. About 1.5 hours into the journey, Brian called me with some news that elevated our day to comically tragic. HE LOST HIS CAR KEYS IN THE FOREST. Brian and Chris needed us to rescue them at Steep Falls, Maine—but first, we needed to retrieve spare keys at his house. To make matters worse, he didn’t know if the spare would actually work. And so the day lengthened, terrible news for the extremely cramped passengers in the mini-van.

Brian Finds the Payload Near Steep Falls, ME

 

Finding Brian Jennings

I kept falling in and out of consciousness, so I don’t know how many hours it took to get to Steep Falls. All I knew was that Greg’s parents were troopers and that they were picking up the pieces of our doomed journey like champions. We found Brian and Chris in a convenience shop in that small town around 9:00pm. Greg excitedly began downloading videos from the payload cameras to his laptop while Greg’s dad, Brian, Chris, and I drove down the road to Brian’s car. And guess what? The spare keys didn’t work! Thirty minutes of referencing the Internet for solutions produced no results, and so we reached a conclusion. Brian and Chris would need to have the car towed to the nearest Saab dealer on Monday morning (it was Saturday night then). Ultimately, Greg’s mom took Brian and Chris to a 24-hour business location nearby so they could wait for their dad to come pick them up two hours later.

Back in the convenience shop where Greg worked on his laptop, the big moment had come. We saw space, and it was beautiful. That moment of incredible happiness almost brought a tear to my eye, and everything we experienced had suddenly become worth the hassle. Everyone was impressed, including the guys who worked at the convenience shop. They thought we were geniuses!

UPDATE: See comments for more info on what happened to Brian and Chris!

 

Side Story: Brian and Chris Encounter the Police

Back when we were driving to Brian’s house, Brian and Chris ran into some trouble of their own. After an unsuccessful search for their keys, they left the payload at the side of their car (!!!) and hiked toward “downtown” Steep Falls. As this happened, a local police officer was alarmed that a mysterious New Hampshire car with an odd block of foam and an axe in the back seat was parked in an unusual forest-side location. Recently, as Brian later learned, a man in a New Hampshire car parked at the side of the road and beat up a woman, causing some scandal in that small town. The memory still fresh, the police officer ran Brian’s license plates and managed to get in contact with Brian’s mother at his home. She told him the story, and then the officer took the payload with him and made his way to the convenience store to question Brian. All turned out well eventually, and the officer left with some advice for Brian and Chris when he learned they walked on private property: “You know people have guns here, right?”

 

From Maine to Troy, NY

I scarfed down a sub at the convenience shop before we set off for Troy, NY. I happily report that the trip was completely smooth, no troubles, albeit incredibly long. At a rest stop, I was thrilled to see a shooting star for the first time in years, a nice little sign from the Universe. We arrived home around 2:00am, where I quickly fell asleep after 38 hours without sleep.

 

The Work Continues

Right now, Greg and I are working on creating the 360-degree immersive panoramic video. Once that happens, I will execute my plan for garnering tremendous attention from the Internet and national media. I am confident that the end result will be so unique and exciting that we will inspire hundreds of thousands about space. I will keep you posted!

 

A 3D Trajectory of the Payload Travel

Notice How the Payload Rapidly Falls Before Entering Jet Stream