Looking Back at Fleet Commander

It's been a little over six years since I started doing research at the Electronic Visualization Laboratory (EVL) at the University of Illinois at Chicago and the first major project I did would eventually be known as Fleet Commander.

With Star Wars: Episode VII out and EVL's Cyber-Commons touch wall again configured in a way that would make it easier to bring this back, I thought I'd look back at the project that eventually led me to graduate school.

While there have been several iterations on different touch devices, the most famous is the 20-foot touch wall version (video above). Not only did it set the tone as I finished undergrad and continue to grad school, but those wall radial menus and touch gestures would eventually figure into my Master's thesis.

Fleet Commander basically started with the following points in mind:

• I had just taken Video Game Design (Spring 2009) creating a Foosball game for a multi-touch table
• I wanted to originally create a strategy game for the class - team decided otherwise
• I'm employed under the Research Experience Undergraduate program to continue development of multi-touch applications on the TacTile system
• and well Star Wars!
• The first project for the video game design class was to create a bug squish-like game for a multi-touch table so naturally I turned it into an asteroid smash game with an 'asteroid chase' Easter egg complete with a TIE Fighter.

Asteroid Smash (2009) - Honestly surprised that 1) I still had the source code on my laptop in 2015 and 2) was able to get it running in a few minutes

Concept

I remember sketching out the original draft of 'Planetary Defender' - fleets above fighter for control of the planet below where the player would have both the ground base and the fleet at their disposal. Maybe a bit ambitious for a single fourth year computer science undergraduate with a semester of video game design (Zooball) and one software design project (text-based go-here, pick up item, unlock door type game).

Fleet Commander would eventually be the result of 'phase one' of the game which focused on just the space battle component.

Planetary Defender Concept Art
(Honestly I'm starting to regret not keeping the original capital ships I drew in Paint)

The core idea behind Fleet Commander wasn't the ships or even the gameplay, but the user interface. I've basically built my entire grad school career on user interfaces and human-computer interaction. At least that was the idea to justify this as an undergraduate research project.

Keep in mind this was 2009 (Angry Birds wouldn't be a thing until the end of the year) and I was thinking: I have the 52" touch screen, 100+ touches, no distinct user identity, no 'correct' orientation on the table, and how to fit a strategy game with touch clicks and drags (no keyboards or right-clicking).

A group of us that built TacTile and/or took the game class in 2009 would eventually publish Design Guidelines for Multiplayer Video Games on Multi-touch Displays.

Building The Fleet

Development at a Glance

• 7 months total development time
• Spanning summer 2009 to July 2010
• Mostly during the academic year (My senior year in college)
• Small updates through 2013
• 1 developer (me)
• ~13,000 lines of Processing / Java code
• Spread across 31+ classes
• 100+ images
• 49 sound files
• 1 3D object (Sphere for the Death Star with 3 layers of textures - surface, light, and shadow)
• Originally designed for a TacTile, the first multi-touch HD (1920 by 1080) resolution LCD table using Frustrated Total Internal Reflection (FTIR)
• Microsoft Surface 1.0 at the time was projector-based and only 1024 by 768

• Ported to the Cyber-Commons multi-touch wall using a completely different touch system
• ~20-foot wide PQLabs multi-touch overlay (Largest one ever built at that time)
• Wrote the Omicron Input Abstraction Library to manage this
• Updated to support a linear array of speakers located above the wall
• Using a resolution of 8160 by 2304

Structure

The primary purpose of Fleet Commander was to design a native multi-user, multi-touch strategy game. But it also served as a test bed to really explore all those data structures, software design techniques, and algorithms I had read about or was casually mentioned in class.

I really went all out with the object-oriented design. Basically every object that needed to interact using touch was derived from the same 'Touch Object' class. Everything from the ships, buttons, fighters, and superweapons all had the same base class. The Mon Calamari Cruiser, Imperial Star Destroyer, Corellian Corvette, and the Carrack Light Cruiser all derived from the Capital Ship class.

All the fighters flying around are controlled using a basic flocking algorithm, well maybe just the follow aspect. I don't think I really wrote a good implementation of a flocking algorithm until my third attempt - which was for a project for the GPU programming class (and yes if you look really closely, that is the A-Wing sprite I drew for Starfighter Battles Part III.

The aiming mechanic does use a look-ahead method by calculating the velocity vector of the target and the speed of the laser - classic shooting game physics. Again first time I really implemented this kind of AI behavior.

Interface

Radial menus and multi-touch widgets and gestures make up the bulk of the user interface. Buttons are multi-touch aware and track multiple simultaneous touch points. Touch drags are used to position objects - even using all five fingers to launch 5 ships at a time (although a bit difficult to perform). Capital ships can even attack specific targets using a press-drag-to-target or using two fingers to do a ray cast.

As I've already mentioned the techniques here basically setup the basis of my master's degree. The user interface design would appear in my master's thesis as well as the SAGE2™ Scalable Amplified Group Environment.

Sound

Basically all the audio was (obviously) lifted from Star Wars games (If you must know: Shadows of the Empire, X-Wing Alliance, and Empire at War). As someone on YouTube mentioned, there are Star Trek: The Next Generation computer interface beeps in there.

Keep in mind this was written in Java on Linux in 2009. Anyone who has worked with JavaSound in Linux at the time knows how much of a pain it was to get a working sound card driver. Even then you started running into issues with file types, bit rates, and all that fun stuff.

The final solution was to stream sound events to an external SuperCollider server which would handle all the audio work. This also had the advantage to allow us to localize sound effects across the seven speakers along the top of the Cyber-Commons wall.

The one part of the sound manager that I was particularly proud of was the Dynamic Music Manager which basically played/looped/switched music files based on the state of the game. The notable cues were 'Rebel/Empire enters first', 'Ion Cannon/Death Star fires', and 'The Trench Run'.

While filled with sounds and music copyright of Lucasfilm, how I pulled them all together really added to the overall feel of the game. Really going to miss that if I ever end up pulling the Star Wars theme out of it.

The Video

Making of

So we've had Fleet Commander running on TacTile and the Cyber-Commons wall for quite a while and at some point my academic adviser said we should make a quick video of it. Not sure exactly what prompted this - maybe as just as an example of multi-user interaction on the touch wall that was more complex than just the paint app.

I think we shot this video rather fast. Got as many students that happen to be in the lab that day and got them in front of the wall to play it. Fairly sure more than half of the people in the video had never actually played the game before.

If I remember correctly, the final dolly shot was done with our resident video expert sitting on a wheeled chair being push by our audio expert. I know there's a picture of that - will have to find that someday.

Reaction

I believe my academic adviser did a quick edit and threw it on youtube and sent it to Kokatu. I believe the intention was to get it out there and let people knit pick every detail out of it. I think after a couple of weeks it was at 500,000 views.

I think it was the next day I got two separate emails: one from Lucasfilm and another from the creative director as Lucasarts. Both essentially expressed interest in it and asked if I wanted to come down to San Francisco and talk about it.

By chance I was already flying to San Francisco on EVL related business to demo our SAGE software so I was able to stop by and have a look at the Letterman Digital Arts Center where Lucasfilm and LucasArts operated.

Yoda statue outside the Letterman Digital Arts Center in the Presidio, San Francisco.

Really cool to see so many matte paintings and props ILM had done over the years. Maybe if I wasn't just starting graduate school (and after the fact Disney acquiring and shutting down LucasArts the following year) maybe it could have been interesting to work there. With the new ILMxLabs in 2015 and the fact I've been doing virtual reality since 2012 who knows?

Things I Would Have Done Differently

Artwork

As I mentioned in the concept section, I'm staring to miss the original artwork I had done in Paint for the capital ships. I think I just wanted it to look better - especially when corners would start breaking off. That was a bit of a pain in itself since I had to make and align multiple images by manually specifying coordinates to make that work in Processing.

The big part where I got lazy was trying to draw my own space stations - as so many have pointed out on YouTube, I screen grabbed from Empire at War. I'm a Computer Science student and not an artist - although this was a few years before I would start The Enterprise. Even now I'm not sure I'd have the 2D/texturing skills to pull off something that looks good. A majority of Enterprise is just material colors and not textured (The Bridge being the one major exception).

And really, would I want to stick with similar artwork I've been tossing out since 2002?

Star Wars: Starfighter Battles III (2002) - Written in Klik & Play

Unfortunately the Empire at War art was really recognizable and had a bad tendency to overshadow the project. But I really wanted to get a space station in there that wasn't the Death Star. Besides where in the original trilogy did you see a space station that wasn't the Death Star? I suppose there are plenty of examples that have popped up in the X-Wing series of games, but Empire at War did have a distinctive design for the Alliance and Imperial space stations.

The Video

Honestly I've never been crazy about the 'famous' Fleet Commander video despite 775k views as of 11/16/2015. It was such a quickly shot video with next to no editing. My academic adviser at the time really wanted to get it out there as quickly as possible knowing that everyone would nit-pick the hell out of it. I was probably only one of two people in the video that had ever played the thing before.

In particular I feel bad for one particular student in the video since she always got flack in the video. She's since graduated and gone on to better things and probably never looked back at the video (Thank the Force). But maybe I'm just reading into it too much.

As previously mentioned there were plenty of good things that came out of that experience, but still.