Apr 18 2018

The Pointy End of Camera Matching

Hello everyone. My name’s Stuart Attenborrow, and welcome to the semi-regular update on camera matching!

This topic has been covered extensively in the past so I’ll try not to repeat what you already know. However, for those that are new to the project: camera matching is a term we use to describe a range of techniques that all involve using the original in-game stills to position objects in 3D space. We do this to ensure we preserve the scale, position and shape of every element in the game, and it is the most time consuming part of the process. Unfortunately it’s also the first stage of the process, thus creating an immediate bottleneck when we start work on a new area.

Luckily, this year has seen a fundamental shift in the way we’re approaching camera matching. If we were to continue matching each frame by hand, the project would likely never see the light of day. And so we’ve changed tack. Again.

A few of our supporters have offered great suggestions regarding ways of automating the process, including a topic known as photogrammetry. This involves taking lots of photos of an object and using software to generate a 3D model complete with textures. Some examples of commercial software that can achieve this task include: Autodesk ReMake, Agisoft PhotoScan, Reality Capture, or something a bit more obscure like VisualSFM.

Source:
http://openmvg.readthedocs.org/en/latest/_images/structureFromMotion.png

It’s never as easy as it sounds. The biggest problem we have with automating the matching process is that it requires a lot of source imagery to work. Riven might seem like it has a lot of images, but most of the time they’re too far apart, too dark, too compressed, or just plain too small to be of any use. Another small but not insignificant issue is that some areas in the game defy the laws of physics. An interior may end up being larger than is physically possible from its exterior.

This leaves us with a couple of options. Either we announce the end of Starry Expanse and our crushing defeat in the face of insurmountable odds, or we put our thinking caps on. We preferred the latter. To explain what we’re doing now, let’s first go over a brief explanation of what photogrammetry is all about.

There are many stages to a photogrammetry pipeline. The first is feature detection, where contrasty or unique shapes are determined in each image. The second is feature matching, where each image is compared to every other image to determine common features. Next is the sparse reconstruction stage, where 2D becomes 3D. This results in a number of points in 3D space representing the subject, and a number of camera positions that represent where each image was taken. Assuming each stage completed successfully, a final stage can be performed where the sparse reconstruction is used to generate a dense point cloud of the subject. In short, there’s a ton of math involved, created by some very intelligent people (thanks everyone!).

The images in Riven usually fall down at the sparse reconstruction stage. There aren’t enough matching features found in enough different images for an automated photogrammetry pipeline to rebuild the 3D scene to a good standard (or at all). To solve this issue, we’re manually performing the feature detection and matching stages ourselves. Whilst there are a lot less matching features between images, the quality of the matches are far better. This means the reconstruction stage is successful, we get a 3D scene of a few key points, and most importantly all of the original camera positions!

It sounds like a lot of work (and it still is), but it’s an order of magnitude less work than the previous method. Plus it results in a much more accurate outcome. At this point we block out the scene using a simple mesh (you may be familiar with the greyscale landscapes we’ve shared previously). We use the key points as guides, and each camera to align the geometry. This can be handed off to the modelers to create the final assets.

My involvement so far has focused on matching the full motion videos. On the upside, there’s a lot of frames in each video to match, but on the downside the videos are heavily compressed. For the following maglev ride I’ve had to match both directions and merge the result to ensure the location of the supports and docks are accurate.

In addition to this, I’ve been working on a way of incorporating a dense point cloud reconstruction process into our pipeline. Since we’re manually matching the images we haven’t been able to use the same methods employed by the software mentioned above. Preliminary results are very promising. This makes the blockout process semi-automated and greatly reduces the time spent creating the initial geometry. Plus we get textured meshes from the match!

 

To put this task in perspective I’ve put together a little graph. I think the results speak for themselves.

With the camera matching process taking a huge leap forward, our modeling team won’t know what hit them. Expect more great things to come. If you’d like to hear more about what we’re doing, have any questions for us, or if you just feel like a chat, head on over to our new discord server!

Finally,  a big welcome to our newest member Cameron Craig, who’ll be assisting with the creation of our all-important characters.

 


Apr 1 2018

Voice Over: A Sound Decision

(Editor’s note to the reader: This is a post we published on April 1st! You can do the math. Enjoy!)

Hello everyone! Hollister the audio guy here. Happy Easter!

As many of you know, the question of how to handle the audio in our game has been a hot topic within the team. We’ve gone back and forth about whether or not we want to just use the original audio, or if we should perhaps see about recreating it, as we are doing with the models.

The cutscenes are a great example of this. We could just use the original audio, but since we’re creating new character animations, the old audio, aside from being considerably low-quality, wouldn’t line up perfectly with the new animations.

So, this has been a point of contention within the team for a while, as everyone has different opinions on it. Luckily, I’m a very driven person, so I took the initiative and went ahead and used your incredibly generous donations to hire some equally incredible voice over talent.

I haven’t talked to the team about this yet, but I’m completely sure they’ll be on-board. Now, onto the good stuff! We’ve already recorded some of the cutscenes, and I’m eager to showcase them.

The first one is the cutscene of Ghen’s friend talking to you in the chamber you teleport to:

Next, we have Ghen himself, delivering his monologue about his son, who I presume is Atreus:

Then onto Catherine’s monologue when you kidnap her from Moeity:

And finally, we decided we can’t leave out the fan-favorite of Ghen singing, so we decided to have our very talented actor record that as well:

I hope you enjoyed this presentation. Stay tuned for more audio updates!
I’m so happy to be such a big part of bringing Myst 2 to life!

Take care,

Hollister
Sound Engineer / Composer, The Starry Expanse Project

April Fool’s!


Mar 2 2018

Cool Under Pressure: The Boiler

Happy belated New Year from The Starry Expanse Project!

It’s been a while since our last update, but don’t let that fool you into thinking we’ve just been sitting idle. Over the holiday break we took the opportunity of the downtime to  critically review our pipeline, and get all our ducks in a row to be as productive as possible in the new year. We’ve restructured our team to be more autonomous and efficient, and we’ve been hard at work on camera matching.

We can’t wait to share more of our plans for 2018, but until then here is some work in progress from our artists Leonard Schölch and Nick Mower (hey, that’s me!). It’s Boiler Island’s namesake – The Boiler!

2018 is going to be a huge year for us, in more ways than one. We’ve got plenty of surprises in store. Stay tuned!

Welcome to our newest team member, Stuart Attenborrow – a multi talented software developer.


Dec 7 2017

Chipping Away: Jungle Island Terrain

Whether you’ve been following us for some time or just discovered us recently, it’s no secret that reproducing the environments of Riven in a modern game engine is a considerable undertaking, and of all those tasks, terrain is by far the most time consuming.

Historically we have avoided committing too much development time to terrain because in many ways it is secondary, and just wraps around more important features of the environment. Our priorities have always been to first block-in the navigable areas and interactable objects, followed by large set dressing, and only after that was complete would we start on terrain.

We’re now at a stage in the project where large parts of the game are ready for a detail pass, and so our intrepid 3D artist Jonas Becsan has been hard at work sculpting out the terrain on what is inarguably the most challenging location in the game – Jungle Island!

We’re eager to share more of our workflow with you, so stay tuned for a more technical breakdown of our process early next year. Stay tuned!

Note: Edited 2017-12-11 for clarity


Oct 31 2017

Happy 20th!

On this day, twenty years ago, to much anticipation, Riven: The Sequel to Myst was finally released to worldwide critical acclaim. Two decades later, our project to remake this game has become a symbol of how much it means to all of us on the Starry Expanse team. Riven was the high-water mark of the Myst series.

Twenty years of adventure gaming, puzzle solving, and online community discussion… Now, here we are, at the tail end of 2017. So much of the game development and computer graphics industries has changed in the twenty years since 1997, but we are honored that someday we will be able to bring this amazing experience to new players in order to continue its legacy.

To mark the occasion, the team would like to share something that we have been working on recently. We are always looking for new ways to share our work, and although we currently have no plans to publicly share a build of the game, we may have the next best thing:

(View may take a minute or two to load.)

Please note that these panoramic images are a work in progress and do not represent the final position of navigation nodes in realRiven, and neither do they reflect the final quality of the game. They are the result of our experimentation with Unreal’s stereoscopic capture feature, and while they are areas you have likely seen before if you are a regular reader of this blog, we are excited to have been provided a new way to show our environments, which seems fitting for the occasion.

 

Happy 20th Riven-der-Weende!


Oct 7 2017

A Community Update

We’d like to extend a big thank you to all who have contributed to our navigation design discussion over the last month, whether it was here on the blog or on social media. We were blown away by the response, and it was fantastic to see the ongoing positive discussion and constructive criticism within the Myst community.

In the past we’ve been hesitant to publicly discuss some of these challenges, but the response was so gratifying that we have taken some time to update both the forum and our subreddit /r/StarryExpanse (previously gathering dust this last year and more) to help facilitate more communication between the team and those who are following the project.

The new look of our community forum, be sure to stop by and say hi!

We hope to discuss more design considerations with you soon, and if you’ve not had a chance to tell us what you think of our plans for the Boiler Bridge, there is still time to join the conversation here, on our forums, subreddit, Facebook or Twitter. We’re all over the place!

The new and improved /r/StarryExpanse, created by Hollister Starrett.


Sep 7 2017

Crossing Over: Boiler Bridge

In converting a classic point-and-click adventure game like Riven to real-time 3D, many systems need to be reworked or tweaked to make the best use of the player’s newfound freedom of movement. Some, like control schemes for walking around in 3D space, have been tried and tested in many other games and we have a wealth of playtesting data to draw upon when making design decisions. Others, like the following example, can be more difficult to elegantly solve.

This is the bridge that spans the expanse of ocean between Boiler and Temple Islands, and we’ve cleverly called it ‘Boiler Bridge’. This bridge is very, very long.

In the original game, the player can cross the bridge in a trivial 9 clicks (or just 1, if using zip-mode). At our current player speed in realRiven, it takes approximately 45 seconds. This is a really long time to be holding down a button and staring forward, especially since the exploratory nature of the game means new players will be crossing multiple times.

The walk to Boiler Island, at three times the regular speed.

This is not the only example of long distances made difficult in real-time. Boiler Island also has several lengthy sequences in pipes and ducts that have the same problem. Luckily, these only need to be traversed once.

We have explored multiple methods to speed up the journey from Temple to Boiler without disrupting the player experience. Cutscenes, teleportation, and simply upping the player speed have all been considered and rejected for the same reason – it interferes with the player’s freedom of exploration and/or feels too ‘gamey’.

Surely a pontoon bridge would be safer.

We are currently exploring a way of cutting the travel time in half by moving the bridge underneath the player as they cross. This gives an appearance of moving at regular speed while at the same time moving towards the distant environment at double speed. Our newest team member, Hollister Starrett, has been hard at work on a prototype of this feature.

We hope to playtest this system alongside our navigation mechanics very soon, but in the meantime we are interested to hear any ideas you may have to solve this design problem. Have we missed something blindingly obvious? All ideas are welcome!


Aug 3 2017

Mysterium 2017

Mysterium, the annual convention for all things Myst, is upon us once again.

As is traditional, we’ve been hard at work preparing a demonstration that showcases some of the things we’ve worked on throughout the year.

(Updated) Our presentation occurred on Saturday, August 5th at 4pm EST.


Jul 8 2017

Wrangling Frames: Matching Animation

It’s no secret that here at the Starry Expanse Project we’re sticklers for accuracy in our reproduction of Riven, and at the center of this endeavor is camera-matching, the placement of cameras in a scene that represent individual shots from the original game. This process isn’t exactly complicated, but it can be finicky and requires a lot of patience to get the best results.

There is a lot of animation in Riven. Putting aside the live-action character performances, there are buttons, levers, bridges, elevators and more. We use the same principles to reproduce these moving parts.

Camera-matching animation is relatively straightforward, as we use the viewpoint cameras we have already set up. Most animation files in Riven encompass a small section of a shot, so we have to carefully position them to make sure everything lines up. Once it’s set up, we can watch the animation in our 3D viewport and animate accordingly.

The fact that the original Riven was created in a 3D software package is incredibly helpful to our animators. These programs allow artists to move objects in one axis, or rotate them precisely to a fraction of a degree. The original Riven team used these tools as well, which means we usually can take for granted that if an elevator is moving vertically, it will be moving perfectly in one axis only. Likewise, if an object is rotating 180 degrees, it’s very likely to be moving an exact 180 degrees.

One interesting challenge that we face is to find a way to capture the feel of the Riven animations with a variable frame rate. All of Riven’s animations play at 15 frames per second, which is half of the 30 fps standard for games animation, and much smaller than the 60+ fps that we expect modern games to achieve. The difference in the feel of the animation is definitely noticeable.

We hope this delve into our animation pipeline has been enlightening, and if you have any questions please leave a comment . There are still many challenges for our animation team to overcome (character performances in particular), and this will be the subject of another post in the near future.


Jun 10 2017

Stacking Shelves: Gehn’s Lab

There are several areas in Riven that give players a look behind the curtain, allowing them a fuller glimpse into antagonist Gehn’s personal goals and motivations. One of these is his lab on Boiler Island. Its tables and shelves are full of objects that, if examined closely, will provide answers to many of the player’s questions.

Over the last few weeks, our artists Francois Hurtubise and Nathaniel Grove have been hard at work bringing this area to life. Here are some examples of work currently in progress.

A disassembled Ytram trap

Tools for preparing poisoned dart cartridges

We’d also like to extend a big thank you to all those who contributed to the discussion last month! We were overwhelmed by the response, and enjoyed reading through your suggestions. The passion and attention to detail our fans bring to this project continues to motivate us to do our best possible work. We’re looking forward to discussing other important topics with you all soon.

Drawer contents coming soon!

Welcome to our new team members:
Paul Petre (3D Artist)