Over the past two years, as our digital twin feature set has evolved, a clear requirement has emerged: users of our spatial intelligence platform OKO need a seamless way to ground their custom assets in real-world environments.

One of the most widely known and accurate representations of the real world is Google Earth, which thankfully also provides an API to access its dataset via 3D Tiles – an open standard defined by , the platform for 3D geospatial.

What are 3D tiles?

Cesium authored the , which has been adopted by various providers, notably Google and, of course, Cesium Ion.

In simple terms, a 3D tileset is a JSON file structured as a tree of nodes. Each node contains information about a specific region of the world and its geometry. Alongside bounding volumes, coordinates, and detail levels, these nodes can contain either GLTF data, point clouds, or Gaussian splats – the fundamental building blocks used to represent 3D environments.

As an example, an application can traverse the tileset tree by evaluating node bounding volumes and progressively refining into child nodes as the camera approaches an area. This allows highly detailed geometry to be loaded for nearby regions, while larger, lower-detail tiles efficiently represent distant areas

Explored approaches

We identified three approaches to implementing 3D tiles:

1. Add support for CesiumJS as a rendering engine. OKO was designed from the outset to support multiple rendering engines, including PlayCanvas and BabylonJS. We could add CesiumJS as another option, allowing us to provide our scene data directly to the Cesium rendering context. However, this is a relatively heavy-handed solution for supporting a single feature.

2. Run CesiumJS “underneath” OKO. The two contexts are then rendered in such a way that they appear as a single scene. However, this approach requires transformations to remain synchronised between the two contexts and introduces a natural separation between them. Bridging this gap would require additional complexity, particularly in scenarios where users wish to modify or replace portions of the world geometry.

3. Pull the 3D tileset data directly into OKO. In this approach, OKO would parse and render the tileset data within its own rendering context. While this avoids maintaining multiple rendering engines, it adds complexity around tileset parsing, rendering, and the mathematical overhead of ensuring the correct data is where it needs to be.

After a brief investigation, we chose option three: importing and rendering the 3D tileset directly within OKO. This approach allows us to reposition the tiles to fit our scene while minimizing disruption to existing functionality.

The implementation

Typically, a 3D globe application such as CesiumJS or Google Earth renders the globe as the central point of interest, transforming other objects (including the camera) to its surface and orientation. This is effectively a globe-first approach, which makes sense for these applications since the focus is on the globe and other objects are positioned in relation to it. 

We took a different approach. To avoid introducing complexity into our multiplayer and cross-platform systems, we kept the coordinates scene-first and instead transformed the globe to sit beneath the scene, placing the globe’s point of interest at Cartesian 0,0,0. This means the space can be rendered in our other deployments, including Unreal and Unity, without any further modification to transformations, keeping the cross-compatibility simple.

The following describes the steps taken to facilitate this.

Loading the correct tiles for the geolocation

provides the necessary functionality to create a viewport that accurately represents the camera’s position and orientation. We create a MercatorViewport object using the space’s geolocation and the in-space camera’s transform. This allows the library to fetch tiles for the correct location and view frustum, while handling the necessary level-of-detail switches. From there, we hook into tileLoad() and tileUnload() to show or hide the tiles whenever the library signals a change.

Positioning the tiles within a globe

Each 3D tile node contains information such as transforms, model data, and bounding volumes. The GLTF data is parsed into texture, vertex, and index buffers as-is, and positioned within a parent entity using its ECEF (Earth-Centred, Earth-Fixed) coordinates. 

Note that no rotation is applied to the individual mesh entities at this stage; their meshes are already authored relative to their position and orientation on the globe. However, within our own data structure, we do store the tile’s bounding volume. Crucially, this includes the rotation required to match the orientation of its mesh – a detail that becomes important later in the process.

Positioning and orienting the globe

As tiles load in, we identify the one that best represents the space’s geolocation – the “primary tile”. We apply the inverse of its ECEF coordinate to the globe entity, and the inverse of its oriented bounding volume rotation. This rotates the entire globe so that the primary tile sits at the top.

To account for mountains and general elevation offsets, we further adjust the globe’s Y coordinate by the primary tile's bounding volume extent. Note that this can give unexpected results if the tile happens to contain buildings or other real-world geometry. As more detailed tiles become available, we continually reapply this algorithm to keep the primary tile as close to Y=0 as possible.

Separately, we perform a one-time rotation of the globe so that north aligns with -Z in our scene. This ensures that the globe agrees with other mapping systems, such as our 2D satellite map. At this point, despite using separate libraries and data sources, both the satellite map and the globe’s 3D geometry perfectly align.

With just these steps, the scene contains the relevant geometry at a 1:1 scale. The user can walk around the scene (without collision) and add assets using accurate, real-world coordinates.

Handling lighting

Google’s tileset contains unlit GLTF models without normals. Instead, their textures bake in the real-world lighting captured during photography. 

To allow for atmospheric effects like time-of-day, we needed to calculate our own normals, operating under the assumption that we wouldn’t be overriding any existing shadowing. 

This was trickier than expected. Standard methods of calculating normals include the tile mesh skirts, which cause visible shading along the tile edges. 

To fix this, we use angle-weighted vertex normals to smooth the normals across what can be chaotic meshes, while separately using breadth-first search boundary-edge detection to exclude the mesh skirts from the normals-averaging process.

Conclusions

As the globe is treated as just another entity within OKO, we can apply all of our existing scripting and workflows to it. This opens up some interesting possibilities:

• Transforming to another geolocation:  By grabbing the ECEF coordinates of a new geolocation we can interpolate the globe’s rotation and scale. This creates a smooth zoom-out effect before rotating and dropping the user into the new destination.

• Adding multiple globe entities: We can technically position two real-world locations side by side. For example, we could place the Eiffel Tower right next to Big Ben, even fetching city blocks and aligning them to create unique mash-ups with minimal code changes.

One important technical consideration is the curvature of the Earth. As a user moves hundreds of kilometers away from the central origin point, the world geometry will naturally begin to tilt beneath the flat ground plane.

While our current projects don’t require users to travel this far, we can manage it by re-applying the globe’s transforms every 100 kilometers based on the user’s position. Since users are already accustomed to level-of-detail shifts as they move, this adjustment to the globe’s underlying transform would be hardly noticeable. 

By treating the globe as a dynamic asset within a scene-first architecture, OKO handles massive real-world data without breaking cross-compatibility or multiplayer synchronization. This shift allows users to layer custom interactive elements, multi-user mechanics, and real-world geography within the exact same coordinate system. This solves the traditional complexity of globe-scale rendering by simply turning the world into just another object in the scene. 

Gabe Blake is a Producer at �㽶��Ƶ. After receiving a Bachelor’s degree in Entrepreneurship from Loyola Marymount University, Gabe found himself at a VR company called Mindshow, where he grew to love the intersection between new technology and storytelling. After a chance meeting with a former Magnopian, Gabe had the opportunity to join the �㽶��Ƶ ranks in 2019, and he hasn’t looked back since! He started as a Production Coordinator working on projects such as Expo 2020 and Remembering, and has worked his way up to leading larger and larger projects, including Fallout and The Wizard of Oz at Sphere.

Tell us about your role at �㽶��Ƶ!

I’m a Producer! To me, producing means doing everything in my power to ensure a project lands on time, on budget, and at the highest quality possible. I do this by immersing myself fully in the world of whatever we're building. By deeply understanding our overall vision, technical capabilities, and creative goals, I can help create order out of chaos, help our teams make necessary breakthroughs, and clear the path so everyone can do their best work.

What three skills are essential for anyone in your role?

Creativity: Producers may not be artists, but we have to be creative. We frequently use our creativity to find interesting ways to make the greatest impact with our limited time, resources, and budget.

Adaptability: �㽶��Ƶ is always on the cutting edge. Every project is new. “Change is the only constant” is a cliche for a reason. In that sort of environment, it’s important to be adaptable and “be like water” wherever possible. 

Leadership: Leadership in production takes many, many forms. I find that in many cases, it takes the form of “alignment manager”. One of my main jobs is to help unite a diverse team of creative and technical leaders under a single vision. Then I make sure the team executing that vision knows exactly where we’re heading and why. And that I have their backs at each step along the way.

What or who inspires you?

I’m inspired by many things (my wife, my friends, my family), but for this article, John Wooden is someone who deeply inspires me. His philosophy on life, leadership, and teamwork is so simple yet so true, it’s incredibly profound. Especially while working on The Wizard of Oz at Sphere, I made it a habit to read a passage from “” every morning to put me in the right headspace to tackle the day.

Of course, my �㽶��Ƶ teammates also inspire me every day. It really is remarkable what we’ve been able to accomplish over the years. Whether it’s widely publicized or hidden from view, we’ve done so many things that I previously thought were impossible. This team never ceases to amaze me!

What has been your proudest moment while working at �㽶��Ƶ?

Easy. Watching The Wizard of Oz at Sphere world premiere with the entire cast and crew. It was a remarkable experience. We had all poured so much of our hearts and souls into that project. To see it all come together at the last moment, to hear a real audience gasp in surprise and then applaud uproariously when the opening credits finally showcased the enormity of the Sphere screen…it was truly magical. I’ll never forget it.

What's the biggest lesson you’ve learnt in your career?

There’s no question we all need to keep honing our craft to stay sharp. But when things get genuinely difficult, and there’s no clear path forward, mastering a skill can only carry you so far. In my experience, the teams that persevere and maintain their grit typically do so because they share a belief in a vision that is exciting, important, and worthy of pursuit. It’s why at �㽶��Ƶ we’re constantly aiming for the stars. This sentiment is best summed up by French writer Antoine de Saint Exupéry: “If you want to build a ship, don't drum up the people to gather wood, divide the work, and give orders. Instead, teach them to yearn for the vast and endless sea."

How do you approach challenges and setbacks?

John Wooden said it best…

I do my best to learn from my mistakes, so I don't repeat them. Even when I fail, knowing I put forth the absolute best effort possible allows me to sleep soundly and try again the next day.

How do you want to leave a mark on the world – personally or professionally?

My goal is to be remembered as someone who acts with kindness and care for others. ‘Nuff said!

If you could wake up in the body of another person (just for one day), who would it be and why?

The incredible pro surfer To have the physical ability to ride those huge waves at Jaws in Hawaii, and to ride them with such style and ease the way he does…that would be a huge thrill!

What are you reading/listening to right now?

Reading

• Mistborn (I just bought the fourth Wax and Wayne book, The Lost Metal).

Listening: 

• For working and focus: Every Hermanos Gutierrez album.

• Stuck in my head: “Vienna” by Billy Joel.

• Fresh find: '“e.t.d.s.” by .idk.

Tell us something that would surprise us!

I love singing! I was the president of my high school Madrigals choir, and was in the Notetorious, a cappella group in college. I still sing every year at my friend’s annual Christmas party, and I love a good �㽶��Ƶ karaoke session!

Google Cloud Next ‘26 opened with a high-energy nod to the garage-born roots of the company, set to the distinct sound of Weezer. As 'In the Garage' blasted out, the visuals took the audience on a journey through Google's biggest milestones – from a humble Silicon Valley garage, through to the invention of the Transformer, and all the way to the world of agentic AI today. The message was simple: while the tech has evolved, the ‘garage spirit’ required to build something from nothing remains the same. 

You can check out the full keynote, with our music video opener, right .

Led by Creative Director Addison Herr, Visual Effects Supervisor Alex Henning, and Executive Producer Daisy Leak, this production served as a high-octane proof of concept for the future of content creation. Here’s a look behind the scenes at how it all came together.

A hybrid human-AI workflow

Traditional production for a CGI-heavy, era-spanning music video would typically take months of rendering, animating, and editing. We completed this project – from initial concept animatic to final high-definition render – in 17 days. 

Art Lead Emerick Tackett and Producer Eugenia Chung steered a massive effort that combined traditional artistry with emerging AI workflows using Nodey and Google AI models to move at speed without compromising the quality bar. 

is our web-based workspace that creates a unified, secure experience for world-building and content creation. We built it to solve many of the challenges we’ve experienced when using generative media and agents in an enterprise environment. 

Here’s how our creative team created the video:

Stopping the tab shuffle

Using AI for video creation often means time wasted jumping between many different websites and apps to generate images, then video, then upscaling them. Nodey brought all those tools, including Gemini and Veo 3.1, onto a single, unified canvas so the team never had to leave the workspace. 

Mapping the journey in record time 

We didn't wait weeks to see if a concept worked. Starting with a storyboard from Google, our Creative Director, Addison Herr, used Nodey to build a full low-res, pre-viz video in just two days. This gave us an immediate proof-of-concept for art direction and provided our artists with a clear roadmap for transitions and shot counts. By locking in the vision early, we were able to move with the speed necessary to complete the entire production in 17 days.

Modular task chains

On this project, we didn't just type a prompt and hope for a lucky result. Our powerhouse team of AI Artists – Frank Capezzuto III, Kenley Lundy, Porter Hodgkiss, Justin Dykhouse, Tony Kwok, Ella Roberts, and Joe Salazar – utilized a modular pipeline to chain multiple models together. They moved from Gemini text to Nano Banana 2 storyboards to Veo 3.1 cinematic motion. If a shot needed a lighting tweak, the artists could simply update an individual node without re-generating the entire sequence. This check-and-balance logic allowed for massive creative iteration quickly.

Directorial control  

To bring precision to the process, the team treated AI as an additional layer within a traditional VFX pipeline. Compositor Brooke Grossmann worked to lock specific start and end frames for every sequence, providing the essential boundaries for the production. By leveraging Veo’s ability to generate motion between these points, the team gained a level of control that allowed them to lock the vision first and iterate easily. 

The workflow was a true hybrid of man and machine: Lead Animator Mike Ryan would take initial Veo generations from AI Artist Tony Kwok and layer his own hand-animation on top of them. Meanwhile, Compositor Brandon McMenamin developed bespoke composition solutions for specific asset edits. This collaborative approach ensured that if a shot needed a tweak, the team could address it directly rather than having to re-generate the entire video performance from step one.

Collaboration at scale 

Because Nodey uses node graphs, every complex task is a "recipe" that can be saved as a tiny file and shared instantly. If a specific shot was proving difficult, an artist didn't have to struggle alone; they could save their node graph and the whole team could jump on it to help refine the logic together. This turned technical troubleshooting into a shared team effort, keeping us moving at pace.

�㽶��Ƶ-grade delivery 

Ensuring the final product was "production-ready" required a specialized technical backbone. Pipeline Engineer Andrew Bueno architected the rendering pipeline, establishing the cadences, folder structures, and naming conventions – alongside key automations – that allowed the team to function at scale. 

Once the creative sequences were locked, Harimandar Khalsa (Upscale Lead) managed the transition to high definition. This involved extensive R&D to optimize settings for each specific shot and rigorous Upscale Quality Control (QC) to maintain the quality bar. Editorial Lead Russel Fong then prepped these refined shots into final sequences for MP4 reviews, ensuring a seamless flow for the client.

This meticulous process, supported by Alyssa Lizarraga (QC and I/O) and Alicia Mirelez (Operations), ensured that while the initial logic was AI-generated, the final output met studio-grade delivery standards. The entire operation leveraged Google Cloud’s TPU-based compute infrastructure to handle the massive processing requirements of the 17-day sprint.

From concept to keynote in record time 

It ������’t every day you’re asked to help set the tone for Google Cloud Next. Telling Google's story – using the very tools they are currently pioneering – was a landmark challenge for our team and the perfect opportunity to use our own toolkit. 

Nodey served as the essential bridge that transformed generative AI into a collaborative production stage. By removing fragmented workflows and providing frame-by-frame control, it ensured the technology remained an extension of the artist’s hand rather than a replacement for it. As we look toward a future of agentic AI, we aren't just making content; we’re building the frameworks that allow human creativity to finally keep pace with imagination.

Ellen is a Lead Technical Program Manager at �㽶��Ƶ. She originally kicked off her career in Public Relations, where she learned the art of storytelling, leaned into her skill for stakeholder management, and kept things running smoothly behind the scenes. From there, she easily transitioned into program management and hasn’t looked back since. 

Tell us about your role at �㽶��Ƶ!

As a TPM, I keep our Product teams aligned, raise blockers, and assist teams in their effort to turn complex technical work into clear, on-time deliverables. 

What made you decide to pursue a career in this field?

I don’t think I followed a straight path into this industry. I’ve evolved from working in a wide range of industries and roles, building a skill set that helped me realise where my strengths lie. I’ve always been a highly organised person, both personally and professionally – my friends and family would verify this through my crazy level of holiday itinerary planning! Ultimately, that's what draws me to program management – the challenge of organising multiple moving parts and having things come together. I enjoy being that bridge of communication across teams and helping them achieve their end goals. 

What have you been most proud of at �㽶��Ƶ?

What stands out for me the most is the people. When I successfully facilitate teams learning something together and creating something everyone is proud of, it’s a great thing to be a part of. So to summarise, enabling my colleagues to smash their jobs is what stands out for me!

What two skills are essential for anyone in your role?

Being a trustworthy leader: Most of the time, TPMs will need to influence teams without being someone's direct manager, so the ability to motivate and align people with a level of authority and trust is key. 

Being an excellent communicator: Delivering clear, concise comms – from Slack updates to meeting structure – is the glue that keeps everyone aligned and moving towards our goals.

What is the biggest lesson you’ve learnt in your career?

The importance of reading what you write and then checking it again. As a dyslexic, this has been my biggest challenge to overcome. I always remember a previous manager gifted me Dryer’s English by Benjamin Dryer, highlighting the importance of language and communication. It has stuck with me as a really thoughtful gift at the beginning of my career and reminds me of a job and a team that helped shape my future.

What’s your favourite thing to do when you’re not working?

I am a big fiction reader, so my ‘to be read’ list is ever-growing! I love an audiobook, so I like to have one on the go alongside a physical book. My physical book at the moment is The City of Brass by S. A. Chakraborty, and my audiobook is Wild Dark Shore by Charlotte McConaghy. Two very different books, but I think Wild Dark Shore is pulling ahead – it’s a great, gripping mystery novel.  

What's a daily routine you have in your life that brings you joy or just gets you through the day?!

I love to cook!  Leaning into my type A personality, I religiously plan my meals for the week, incorporating new and old recipes into the mix. After work, I find it’s my de-stresser to stick on some music or an audiobook and cook up a delicious meal. I’m just back from a trip to Japan, so I have been making some yummy bowls of ramen and using some of the cool crockery and stunning chef's knives that we purchased out there. 

If you didn’t have to sleep, what would you do with the extra time?

If I didn’t have to sleep, I would use the extra time to travel more. Traveling at night with less of the daytime rush and arriving in new places at the start of the day, ready to go, would be ideal. You could also explore cool parts of cities that are normally rammed with tourists in the wee hours of the morning by yourself, which would be dreamy!

If you could have any other job in the world, what would it be?

If I had no economic concerns and was doing something for pure pleasure, I’d love to run my own little cosy bookshop – either selling coffee or with an evening wine bar. I’d call it “Lewellen’s”, which is a combination of my husband's and my names.

What’s your go-to karaoke song?

“It's All Coming Back to Me” by Celine Dion. It’s a bit of a family anthem between my sisters and me, and has been performed as a karaoke trio at each of our weddings without fail! 

Previously, on OKO…

The year was 2024, the industry continued to fall ever more deeply in love with Gaussian splats, and �㽶��Ƶ was no exception. For spatial computing platforms, such as OKO, Gaussian splats were a wonderful low-friction solution for the capture and creation of high-fidelity digital copies of real-world locations.

Given OKO’s cross-platform interoperable nature, supporting the format required a splat renderer for Unity, Web, and, of course, Unreal.

For Unreal, we needed a solution that we could ship in the OKO Unreal plugin, one that would ‘just work’ and be plug-and-play for users integrating OKO into their Unreal projects. After exploring the market for already-available solutions, we found that (at the time) none were mature enough for our needs, and so we took it upon ourselves to build a best-in-class Niagara-based solution.

Niagara was a great conceptual fit for Gaussian splats. After all, at the end of the day, what was a splat but another word for a stationary particle with some fancy rendering? Since Niagara is specifically designed for rendering particles in a user-controlled way; offers out-of-the-box camera-distance sorting and does it all fast (by offering more-or-less full-GPU-side execution), it seemed perfect.

Since those heady days, however, splat counts per-asset have continued to rise up and up, to the point where we were finding ourselves unable to render the assets via our Niagara-based solution (it’s not unusual these days to encounter assets with ten million splats). We started finding upper bounds to the maximum number of splats we could render via Niagara without requiring the user to make changes to their projects. We were capping out at around two million.

Meanwhile, was also emerging as a truly interesting alternative solution to volumetric video captures. Our Niagara solution didn’t support it, and as flexible as Niagara was, the far larger dataset 4DGS required – coupled with the upper bounds we were already hitting with 3DGS – implied that our implementation would struggle here too. So I started exploring alternatives.

Which is how I ended up implementing a non-Niagara-based Gaussian splat renderer in Unreal. I didn’t get as far as supporting 4DGS, but the result is arguably the fastest splat renderer for Unreal available on the market today. You can find it .

Here’s the plugin rendering ~7 million splats at 60fps.

I operated under these constraints:

• No enforced upper bounds on splat counts.

• Be portable (because it’s part of the OKO Unreal plugin, which should work out-of-the-box for any Unreal project).

• Be as good as or faster than the Niagara implementation (because if we’re making something specifically designed for the rendering of splats, then it should be faster).

• As robust as the Niagara implementation, as easy to understand and maintain.

It really started with the observation that ultimately, rendering a set of Gaussian splats is the act of rendering the same mesh over and over, where each has a specific set of attributes (position, rotation, scale, spherical harmonics coefficients, and opacity). Which, from a graphics programming perspective, is best done via instanced rendering, where we issue one draw call for all splats, provide the GPU with the data it needs per instance, and then let it go to town.

The idea was that we could build our own little splat instanced-rendering code path in Unreal, via the OKO Unreal plugin.

To compute or not to compute

Initially, I had dreams of an entirely separate compute-based rendering pipeline for custom Gaussian splat rendering. Something that happened before the main event of the frame, after which Unreal’s normal render pipeline would take over, and then we’d composite the rendered splats on top.

I was quickly disabused of this, though, once I thought about the need for rendered splats to sort correctly. Not with themselves per se (although that requirement would bite me later), just with other geometry rendered during the translucency pass, and also opaques. Since splats are all technically translucent, generating some kind of composite mask to stitch them back into the frame felt like it would be expensive to generate, produce artifacts, and be a nightmare to build.

Conversely, rendering with scene proxies, static meshes and materials is just a lot more normal for Unreal; they’re the building blocks for almost every Unreal-rendered frame. They’re all designed to be part of the standard Unreal rendering pipeline, and so ensuring splats would be sorted and rendered coherently with the rest of the frame would likely be a non-issue if I took that path.

So that’s how I ended up choosing a scene proxy, mesh and material-based approach.

The scene proxy

In Unreal, for any component with some kind of visual aspect to it, there will be a corresponding scene proxy. Scene proxies are the render-thread representation of a component which exists on the game thread, and it’s where the high-level rendering business logic lives (how many draws, what material to use, what mesh, and so on).

Just as our OKO Niagara implementation creates a Niagara component with a Niagara scene proxy, so too must our OKO Gaussian splat component have its own kind of scene proxy.

The Gaussian splat scene proxy was to have the following responsibilities:

• Represent the OKO Gaussian Splat component on the render thread.
• Issue a single instanced-draw command for the entire set of splats via GetDynamicMeshElements (which is called once per frame).
• Render all instances using a project-config defined mesh and material.

Early implementations were pretty encouraging. It wasn’t very hard to write a custom scene proxy that issued an instanced draw call for N splats with the right positions, and we could get a sense of what performance would be like by rendering the same N splats hundreds of times to see how it might scale.

Bolstered by the results, and with perhaps a touch of hubris, I turned my attention to how we could sample the rest of each splat’s attributes into account, to render them correctly.

A note on instanced rendering and custom instance data

So, having committed to an instanced-draw approach in scene proxies using a single static mesh and material, the problem to solve became how I could possibly sample the rest of the splat attributes in a material.

Using the material-based approach solved the problem of making sure the splats got rendered at the right time in the frame, and would be occluded correctly, but the standard material in Unreal ������’t really designed to accept bespoke buffers of other data.

One option here was to store the attributes as texture data, and then sample from those textures in the material. Honestly, I think that might have been the better choice, particularly since the advent of the .sog format (which does exactly this), as it’s more amenable to the idea of streaming splats from the cloud, which’ll need to become a thing if they continue to rise in count.

But that’s not what I chose. Instead, I took a leaf out of Unreal’s instanced mesh component’s book and turned towards custom instance data.

The nice thing about this approach was that, in terms of accessing the data from the material, Unreal does provide an easy-to-use interface for accessing custom instance data, and so it became relatively easy to retrieve it when rendering each instance.

The material

Now I was off to the races. Armed with a proxy and mesh, a material that could access all splat attributes and a bag of dreams, I set about reimplementing the Gaussian splat GPU-side implementation we had already built with Niagara in a normal Unreal material.

First up was implementing SH0 coefficients (which can also be expressed as the splat’s Albedo). They’re just a constant color, really, at the end of the day. So as you might expect, that wasn’t very hard.

Next came the vertex shader logic, handling splat rotations and scale. That was harder, but having a working reference implementation with Niagara really helped. 

It’s also worth noting here that while I was all gung-ho about all the advantages of an instance-rendering implementation over Niagara, there were also some things that Niagara did out of the box that required re-engineering in the material.

One in particular was billboarding. Splats are always expected to billboard, such that the mesh is facing the camera directly (or rather, the normal for the plane we’re rendering the mesh on is the negative view direction).

All sorts of other transformation problems, but billboarding’s working!

So I ended up implementing a manual vertex transformation step in the material to billboard the splat mesh, once the main Gaussian splat transformation algorithm had been computed.

I think I made a mistake…

As you can see, getting the transforms just right was fiddly – it’s the hardest part of the Gaussian splat algorithm to grok for sure. I made more mistakes along the way here than I care to admit, but being able to pare things back to just a single splat rendered side-by-side with the Niagara implementation, viewed in wireframe mode, often pointed me in the right direction.

Paring back to a single splat to compare transformation results was a super effective way to validate.

Once I had a single isolated splat transforming in the same way it did in Niagara, I knew I had it figured out.

But it was about this time, to my eternal shame, that I realized I’d forgotten a fundamental aspect of Gaussian splat rendering. They’re expected to be sorted by camera distance, every frame.

In the image above, you can see how much brighter the dinosaur looks. That’s because, after having that realization (that splats of course need sorting), with no plan how to solve it, I quickly took the coward's way out and made the material additive to get a semi-reasonable frame. But sorting and translucency are hard requirements; without them, you end up with artifacts like this…

A DiNOsaur

Not quite sorted

I was pretty worried at this point. With the splat instance data all held GPU-side in Unreal’s GPU scene (which is essentially opaque across the engine/plugin divide), and with no easy means to invoke a sort of all instances, I wasn’t sure what to do. It wasn’t a problem for the Niagara implementation, which exposes particle sorting options right in the editor UI, but no such thing exists (understandably) in the Unreal Primitive domain.

Sorting on the CPU was obviously out of the question. The splats have to be sorted each frame, depending on where the viewer is. Re-uploading sorted splats each frame would be crazy; the desire to support millions of splats implies it has to be done GPU side, as it’s an embarrassingly parallel problem.

So somehow, I needed to invoke a GPU sort of all splats, weave it into Unreal’s Primitive rendering pipeline, and write the results back to the GPU scene, which ������’t directly accessible across the plugin boundary.

So I dove in to see what I could find.

1. Register your key generation delegate with the sort manager. That’s going to be where you populate the sort manager’s buffers with your sort keys and values.
2. To invoke a sort, call FGPUSortManager::AddTask, providing the sort options you need via the sort flags (like whether to run the sort after PreRender, or after drawing all opaques), and the number of items to sort. That’ll arrange the sort task for you.
3. And finally, consume the results. Depending on where in the frame you asked for the sort to happen in 2, bind to the PostPreRenderEvent or PostPostRenderEvent and use the contents of the sorted buffers.

The entire solution hinges on this guy, so hopefully it won’t be internalized for a while, at least. I imagine it’d be tricky to do so, given the Niagara and PCG implementations already depend on it.

But anyway. While there’s sunshine, make hay. With the GPU sort manager understood and a means to write the sorted instance data to the GPU scene, I had everything I needed and began to implement. 

This is how the algorithm ended up:

For each frame:

1. Call UpdatePrimitiveInstancesFromCompute once per frame.
   1. Which enqueues our lambda, to be executed at the very start of the next frame.
   2. When the lambda is invoked, we use a compute shader to write back all the splats, which were sorted in the last frame, to the GPU Scene.
2. Later that frame, after the PreRender phase, we generate sort keys for the splats using the distance from the camera.
   1. This is where we use FGPUSortManager (specifically the singleton owned by the current world’s FX System Interface).
   2. While our sort keygen compute shader is running, since we’re already iterating through each splat, we take the opportunity to also do splat culling.
3. Immediately after running sort key generation, still in PreRender, we run the actual GPU sort.
   1. Again, relying on FGPUSortManager, we let it do the heavy lifting and run its radix sort, using the keys we provided for the current frame.
   2. This is the sorted data we’ll write to the GPU scene next frame (via step 1).
4. Much later, as part of the normal scene proxy rendering pipeline for translucents, we run a single instanced draw call for each Gaussian splat component in the world.
   1. Which uses the sorted splat data for the last frame, which we wrote to the GPU Scene during step 1.

Which worked! With caveats. Which brings me to the most computationally expensive part of this tale.

You may have guessed, but writing all splats to a buffer, per-frame, especially for large numbers of splats, is super expensive. It also implies having two buffers containing all splat data (which means we’re really occupying twice the memory we need) – one for sorting and then again with the actual primitive data used during rendering in the GPU Scene.

Even worse, both of these time and space costs scale as the number of splats being rendered increases.

In an ideal world, we shouldn’t even need to do it at all; it’s theoretically possible to sort the data in-place in the GPU Scene, but to do that would require engine changes. Since the goal is for this to be plug and play for any project, that's not an option. Ah well. Such is life.

Memory

So, given the plugin-only constraint, I decided to assume I was now using the API the best way I could to achieve my Niagaraless dream (if someone reading this is facepalming about now [because there’s clearly a better way] – get in touch!)

Assuming that the sort-buffer-to-GPU-scene per-frame copy was a necessary evil, I figured the one thing I could still do to speed things up would be to reduce the size of a given splat. Smaller splats mean less data to copy. Less data to copy means faster copies!

I went through about eleven different iterations with my splat packing adventures. I won’t bore you with the details of each (let’s just say friends don’t let friends pack splat scales into bytes). I had started with a simple, naive 32-bit float per-attribute approach, so there was lots of fat to trim. Here’s what I ended up with:

// Pack the data into the attributes buffer. Each set of splat attributes
// is packed in a single float4. The packing strategy is as follows:
//
//---------- Float 1 ----------
//    half - Orientation X
//    half - Orientation Y
//---------- Float 2 ----------
//    half - Orientation Z
//    half - Scale X
//---------- Float 3 ----------
//    half - Scale Y
//    half - Scale Z
//---------- Float 4 ----------
//    byte - Opacity
//    byte - SH 0 (normalized)
//    byte - SH 1 (normalized)
//    byte - SH 2 (normalized)

All together now

So we’re nearly at the end of my tale of misadventures with instanced Gaussian splat rendering in Unreal. I ran out of time to truly dive into the exciting world of 4DGS, but I’m very happy with where this iteration ended up. What exists now in the OKO Unreal plugin is one of the fastest splat renderers available on the market.

Given the speed with which the industry is advancing the tech, I do wonder how long that title will last, though.

My guess is not long. There’s lots of opportunity for rendering faster. This implementation has no notion of LODs, for instance, and there are some really interesting things happening with LODS for Gaussian splats at the moment. Some people are experimenting with . Others are exploring . Others have explored applying more .

Meanwhile, here are some more pretty things to look at (and stats, of course) to give you an idea of where we’re at. In case you’re wondering, these were captured on a machine with a Laptop 4090.

Before – with the naive memory layout.

After – over 50% saved on frame time.

My favourite asset. The gorgeous Library/Dining Room at Sir John Soane’s Museum, rendered as 3.5 million splats, running at 60fps in Unreal.

Thanks for making it this far. If you’re interested in learning more about this implementation, I wholly encourage you to get curious and dive into . Play around and let me know what you think.

Until next time!

Ben Cornaglia is a Lead Technical Designer at �㽶��Ƶ. He bridges the gap between creative vision and technical execution, following a Master’s degree in Game Design and a career defined by complex world-building. His experience ranges from prototyping gameplay mechanics to his time at Rockstar North, where he specialized as a Designer. In recent years, Ben has focused on the intricate architecture of interactive environments, balancing the constraints of game engines with the limitless possibilities of player interaction. At �㽶��Ƶ, Ben is leveraging his expertise in technical design and world-building to help create the next generation of shared physical and digital experiences.

Tell us about your role at �㽶��Ƶ!

I’m a Lead Technical Designer within the design team at �㽶��Ƶ. I’ve been involved in projects like the Daft Punk Experience in Fortnite and the Hoppers VR experience in a technical leadership capacity and as an individual contributor. My role in these projects was to provide a technically-minded approach to design and support the teams with tools, reusable resources and more traditional world-building/level design contributions.

I’m also heavily involved in training and mentorship to help my team upscale in many different areas – from technical implementation to design specifications.

What three skills are essential for anyone in your role?

Logic, curiosity, and communication are essential for a technical designer. 

You need to be able to transform a brilliant and creative idea into a strong, future-proof technical solution. That’s where logic comes into play. How do you take that idea and extract all of its fun and ingenuity out of it, while dealing with technical limitations from engines, hardware, etc?

Being curious is one of the answers. Always be looking around for new technologies, new approaches, try to retro-engineer some of them or use them as inspiration for your own technical proposals.

Finally, there’s no point in having the greatest solution to the greatest idea if you cannot communicate it. Use it to give confidence to your team that this will be done, and it will be awesome. Being able to both convince and teach, and adapt to your audience is key.

Who inspires you?

I have been, and am still being very inspired by people like Simon Sinek and Jay Shetty for their insatiable thirst to understand “why”, both from a humanistic perspective and business applications of learning how to grow yourself in the best way possible.

Some historical figures like Ada Lovelace, the first programmer and scientific mastermind, or Rosa Parks, for her unrelenting fight for civil rights, have also shaped who I am today.

What’s the biggest lesson you’ve learnt in your career?

Opportunities are arguably one of the most important factors of success. But I’ve learnt throughout the years that you can both position yourself to have more chances to find some, or even create your own opportunities. Part of that is being nice, genuine, and honest with the people around you. If you come from a place of love instead of over-ambition and competitiveness, people will give back to you, and you’ll have better chances of reaching whatever goals you have for yourself and the team.

How do you approach challenges and setbacks?

I welcome setbacks and challenges. They usually come from something that you haven’t planned for or not anticipated. I view this as a learning opportunity, a means to get better so that it doesn’t happen again. What’s important to me is that we assess why something has been a setback or a challenge, analyze what happened, learn from it, and document it. It’s equally important to me that everybody around me knows that there’s value in reporting a problem, whatever it is, so we can have a culture of trust and confidence that setbacks and challenges are part of what we do, and we can deal with them.

If you didn’t have to sleep, what would you do with the extra time?

It would probably make me sad not to have to sleep. If I didn't sleep, my cat would miss waking me up at 6 am for food and scratches, and I'd miss this daily ritual too!

If I really didn’t have to sleep, I would probably read more. I’m spending most of my time on the computer or phone, and I have a reading bucket list that only grows as the years go by.

What values or principles do you consider non-negotiable in both your personal and professional life?

Accepting differences is non-negotiable to me. In fact, I believe that our differences can and should be leveraged because they complement one another. I understand that people can be hesitant to engage with what feels unfamiliar, but that uncertainty can be reduced by taking the time to understand it.

Also, I think we must be able to say “sorry”. Being sorry is not a weakness; it’s an acknowledgement. It shows self-reflection, empathy, and willingness to be better. Things happen, and mistakes are made, but to me, a mistake ������’t a problem; it’s when you don’t acknowledge it that it becomes a fault.

What are you reading/listening to right now?

My playlist is a bit all over the place. At the moment, I’m listening to Twenty One Pilots, Bilmuri, Aurora, Alicia Keys, Rage Against The Machine, and Starcadian, to name a few.

My reading time is the Dungeon and Dragon 5.5e rule book and other DnD related material, as my wife is very adamant about me creating a new campaign. Before that, I was reading Mistborn from Brandon Sanderson, which I highly recommend for Fantasy enthusiasts.

What is something you think everyone should do at least once in their lives?

Travel. I haven’t been able to travel far outside Europe, but it’s already been life-changing for me to see how different geographically close countries can be. Even going from France to the UK is an experience.

What’s fascinating about it is also finding all the commonalities between places, what we share, what we aim for. 

I’m very aware that travelling is expensive – for your wallet and the environment – but I think it’s worth doing at least once in your life.

If you could have a superpower, what would it be?

Ha. Probably flying, so I can do all that travelling I mentioned for free. It would probably be very cold, and I can’t fly above a certain altitude, or there’s no air. Risky superpower, but sounds really fun.

Sometimes, we can’t just "paint" details onto a 3D object to make it look real; we actually have to build those details into the physical model. Think of it as the difference between painting a brick pattern on a flat wall (texturing) versus stacking individual physical bricks (modeling).

Usually, we handle these complex tasks using traditional methods such as instancing or sculpting. While these methods get the job done, they come with a few major drawbacks, which can not only make our job more complex but can also have knock-ons to a project schedule.

By shifting to a UV Blendshape workflow – this creates mesh deformations based on matching UV coordinates rather than relying on identical mesh topology – we help solve some of the biggest bottlenecks in the modeling pipeline.

We use the more complex "brick-stacking" route for two main reasons:

1. Real-world physics: If an object needs to break, move, or react to gravity in a simulation, it needs to have a real physical structure.

2. Client requirements: If a client provides us with a specific 3D part or asset to use, we have to build our model around that exact piece to ensure everything fits perfectly, e.g., Lego bricks.

The secret: work flat 

This technique was originally perfected for film projects, where we needed a way to apply intricate patterns – like complex Celtic knots – across dozens of different building shapes. The secret was keeping everything flat during the design phase. 

Think of it like decorating a piece of wrapping paper while it’s lying flat on a table, rather than trying to draw perfectly straight lines after it’s already been wrapped around a box. By keeping our models "flat" and editable, we can apply those lines with greater accuracy and simplicity than traditional modelling methods. 

How it works: a software-agnostic approach

The best part? This process fits easily into any artist's routine who is already familiar with 3D modelling. While I personally use Modo for the initial building, the same results can be achieved using any 3D package – the software doesn’t matter, it’s the methodology that’s important

To make sure the final files work widely for everyone else in the industry, the prepared assets are then taken into Maya to “wrap” the shape before moving it back into your package of choice. Here’s why that’s a win:

• No expert knowledge needed: We use a small, automated script to handle the heavy lifting. You don’t need to be a tech wizard or a Maya expert to get great results.

• Industry standard: By finishing the work in Maya, we ensure the files are compatible with almost any other professional studio or software.

• Seamless fit: It’s designed to be a "plug-and-play" addition to the work we’re already doing, not a total overhaul of how we work.

Before we dive into the technical work, we need a clear plan of attack. We usually start with a concept sketch or, ideally, a base model provided by the client. This acts as our "digital skeleton”; from there, we look at how the object would be built in the real world.

We break the big shapes down into smaller, realistic parts. If we’re building a boat, we don’t just treat it as one solid block; we look at the individual wooden planks and their layout. We make sure the "grain" of the materials follows the right path. For a boat, this means ensuring every plank curves and flows exactly how a shipbuilder would lay them out.

By starting with a realistic structure, the final model doesn't just look like a 3D image – it feels like a real, tangible object. It ensures that when we apply our textures later, they sit naturally on the surface rather than looking "pasted” on.

Once we have our plan, we can start building the actual 3D shape. Think of this base shape as a mold or a guide; everything we build later will be stretched and wrapped over this surface. To get the best results, we follow two main rules: 

1. We build the shape with "seams" in mind (like the seams on a piece of clothing). This helps us know exactly where the different parts of the model will eventually split or bend. 

2. We keep the detail level high at this stage. If the base shape is "chunky" or blocky, the final product will look jagged. If the base is smooth, the final product will look sleek and realistic.

Think of a circle made of straight lines, like looking at the profile of a wheel. A wheel made of many small edges appears smooth, with real-world curves (LOD0). At low resolution with larger spans between the edges, it has flat spots and looks "faceted" or blocky (LOD4). By putting in the effort to make this mold smooth now, we ensure the final model looks high-quality and professional from every angle. Making the future steps far simpler and more successful. 

Once the shape is ready, we create its UV map. Think of this like taking a 3D object and carefully "unwrapping" it so it lies perfectly flat on a table, like a clothing pattern or a flattened cardboard box. This flat map is the most important part of the process. To make sure the final result looks exactly as expected, we follow three golden rules:

1. No stretching (minimal distortion): We make sure the pattern isn't being pulled or warped. If you drew a perfect circle on the flat map, it should still look like a perfect circle when wrapped back onto the 3D boat or shirt.

2. Consistent sizing (uniform density): We keep the scale the same across the whole project. You wouldn't want the wood grain on the front of the boat to look giant while the grain on the back looks tiny.

3. Everything facing the same way (uniform directionality): We make sure all our UV islands are pointing in the same direction. It’s much easier to work when everything is lined up neatly, rather than having to juggle pieces that are upside down or sideways.

By taking the time to make a clean, organized "flat map", we ensure that any pattern we apply later – whether it’s wood planks or metal chainmail – fits the first time, free of any awkward seams or dodgy spots. Extra time at this stage is a solid investment, as it eliminates troubleshooting later and allows our efficiency to compound as the project scales. 

At this point, we’ve finished the hard work of building the core structure. This master template is now universal, meaning we can use it to create almost anything without having to change the foundation.

• Set it and forget it: We keep this base shape and its map exactly as they are. This ensures that every version we create afterwards stays consistent.

• Agnostic building: Because the foundation is solid, it doesn't matter what "asset" we decide to use later. Whether we decide the boat should be made of mahogany planks or carbon fiber hexagons, the system we’ve built handles it all the same way.

• Easy updates: If the client asks for a major change to the overall shape, there’s no need for a total rebuild. We simply update this one master base, and those changes can then flow through to every variation, keeping the project on track.

We’ve moved from "hand-building" to "mass-customization." We have a reliable, unchanging base that allows us to experiment with different looks and materials at speed. This is a middle ground between a manual and instance-based workflow.

Creating our blend shape setup

Phase 1: Clean up and scene housekeeping 

We start by moving the model from the software it was created into Maya. Ensuring that when we import it into Maya, we maintain the original scale. Using the .FBX format is a good way to avoid any conversion issues. For good housekeeping, we want to make sure to set the model up before we perform any actions.

• We "freeze" the imported pattern model to tell Maya: "This is the starting point." It’s like resetting a scale to zero before moving anything around.

• Over time, 3D software keeps a "receipt" of every single click and change. This can make the file heavy and prone to crashing, or introduce unforeseen bugs. We wipe this history to avoid this.

• Centre the pivot to make sure the object's "anchor point" is in the right location. It’s much easier to rotate or move a wheel if the axle is in the center rather than off to the side!

If you are not familiar with Maya, this could seem like a learning step; it’s actually very easy. In the poly modeling tab, we have a set of "One-Click" buttons that handle all of this housekeeping instantly.

Phase 2: Setting up the script command center

We want to keep a few control windows open while we work. Think of these as our command center – they give us all the controls we need and provide a look at what Maya is doing in the background. This allows us to make adjustments on the fly and to control our blend shape. They’re also our early warning system; by watching the live data in these windows, we can spot small errors immediately. It’s much easier to fix a tiny glitch when it happens than to try to find it later when it's buried under hours of work.

We want to open up the shape editor (this is where we control our blendshapes) and the script editor (where we run our script and get real-time diagnostics).

With these open and our model in Maya, we are almost ready to go with the creation of our flattened UV blendshape. All we need to do now is open the flattened UV script (linked at the bottom of the page) into a MEL (Maya Embedded Language) tab within the script editor. This can be done by hitting the + icon and selecting the MEL option to give us a clean workspace.

Next, navigate to the File menu at the top left of the script window and open your script file. You will know it has loaded successfully when the MEL window is no longer empty, but instead displays the script instructions.

Phase 3: Generating the flattened blendshape 

Now it starts to get fun! With our model selected, click on the text body within the MEL script and hit CTRL+Enter. This will run the script. 

If we find ourselves doing this a lot, we can turn this command into a single button on our toolbar. I personally prefer to keep the script window open while it runs; it serves as a "diagnostic screen" that shows exactly what Maya is doing, making it easy to verify the process. To test if it has worked, check the Shape Editor  – you should see a new blendshape with a slider. Moving this slider will transition the model between its various states. 

Now that our blend is ready, we need to extract a flat version to work on. We must use the exact same model we have just flattened so that the size and shape match up perfectly later.

We move the slider that "unrolls" the 3D model until it is 100% flat and presents in the same form as the UV map. Hit a shortcut (Ctrl+D) to make a duplicate; this allows us to work on the copy while the original blend stays safe and untouched. Clear the history, etc., of the duplicate, the same as we did above. This makes it a clean, simple sheet that is ready for us to start designing on. Then export the flat sheet and get to work on our pattern.

By using the same mesh for the flat version, we don't have to worry about things not lining up later. It ensures that whatever we draw on the "flat" version will wrap around the 3D version perfectly, with zero guesswork.

Pattern creation

Back in our DCC (Digital Content Creation) tool of choice, it's time to integrate the specific assets required for the final build. For this example, we’ll use three bespoke wooden planks, provided by the client. By treating these as “existing IP”, it forces us to maintain a recognizable brand identity. A plank is a very simple example, but it makes it easy to visualise the process and start to understand how powerful this workflow can be.

In a traditional approach, this is where we would start playing with deformers or jiggling around verts to shape and form the board, which needs to be done one by one. Working with the assets using these techniques can distort the existing UVs and, most importantly, mean that if we need to swap the plank out later in production, we’d have to start from scratch.

Building onto the flat canvas 

Instead, what we’re going to do is take our flat canvas (the flattened UV mesh) and start placing our building blocks, like wooden planks, exactly where we want them to go.

We look at our reference photos (for example, a real boat) to see how the wood was originally laid out during construction so we can mimic that layout. We need to make all our planks stay within the boundaries of our flat map. This ensures that when we blend the model back into a 3D boat, the seams line up perfectly and nothing looks broken.

Occasionally, a plank might land on a tricky corner or a sharp curve. In those cases, we might do a little bit of "hand-trimming" to make sure that specific piece fits perfectly. It's important to note that, for those outliers, some manual UV and texture work will be needed based on the edits to the mesh. 

We will lose a little of the base element, but with this approach, we are keeping 95%+ of the original rather than a full rework. Even though there is still a bit of manual "hand-fitting" involved, this method is a massive shortcut. We are preserving the original information rather than rebuilding from scratch.

This then gets us to a stage where we can review the layout, get notes, spot errors and plan around the build before we have even got the planks into their final form.

Now that we have all our pieces laid out, the planks are arranged, and the layout is approved – it’s time for the final magic trick. We’re going to glue our new detailed pattern to our blendshape so it can take the shape of the boat. This example only uses three meshes and texture sets, so there is some repetition; this would be avoided with variations and tweaks.

To see how the process works, you can run a very simple test using your blendshape model and any primitive mesh.

Phase 4: The final wrap and diagnostic check 

We start by bringing our pattern back into our Maya workspace (clearing history, etc.). Because we were so careful with our scale earlier, the new pattern drops exactly on top of our flat guide, like a perfectly placed sticker. We did all the heavy lifting on a flat surface; we don't have to struggle with "sculpting" every individual plank into a curve.

In the Maya scene, we’ll use the Wrap deformer to ensure the pattern moves synchronously with the blend shape model. This uses a volume falloff to handle complex 3D shapes, rather than only a flat object.

• Select the pattern, then shift + select the flat blend shape model. 

• Navigate to the Deform dropdown.

• Find the Wrap deformer

• Click the Option Box before it runs. 

This opens the Wrap Options menu, where you configure the tool's behavior and how it interacts with the model. Generally, Auto weight threshold and Volume for the falloff mode will work. These will yield the correct result approximately 90% of the time. If the setup is incorrect, don't worry – these settings can be adjusted even after the tool has finished running and dialed in on a per-mesh basis. 

This is where the Script Editor we opened earlier becomes our best friend. Occasionally, the process might seem like it’s stuck, and without that window open, we’d have no idea why. The tool we use to wrap the pattern has a built-in memory limit, and it’s not always clear where that limit is.

If we try to wrap a complex or dense pattern, the tool might run out of memory. Instead of just failing silently, it sends a warning to our Script Editor saying it has run out of memory. Without the Editor open, the tool would appear to have done nothing, and we’d be left guessing. With it, we can see the exact error message and know exactly how to fix the layout to fit within the tool's limits.

If we hit this limit, we simply break the pattern into slightly smaller, more manageable chunks. The end result is exactly the same, but the tool can handle it much more easily.

Now we have our model wrapped fully to the blend shape, so when we move the slider, the blend will move, and the wrapped model will follow – maintaining its volume and all of the details we built into it. If the manual fixes haven't already been done to the modified edges, we can easily see now where they exist and perform the work as needed (ideally on the flattened version to avoid reworks). 

There can be a bug at times where putting the slider to 0 causes the pattern to break free; setting it to 0.001 will fix this.

At this point, you have your mesh in its final wrapped form; it can be exported and brought into production. You can also create multiple versions of the base pattern with alternating planks or layouts, repeat the wrap, and you now have multiple versions of the boat ready to go in no time.

Once we have our system set up, we can use it to add more than just the "skin" of the boat. We can use the exact same process to build the internal structure, like the ribbing or the frame.

By simply nudging our pattern slightly up or down on the flat map, we can instruct whether those wooden beams should sit on the inside or the outside of the boat's shape. We don't need to build a new system for the interior. We use the same blendshape and simply swap the outer planks for inner ribs. It’s like using the same cookie cutter to make both the cookie and the icing – everything is guaranteed to fit together perfectly.

This layering technique allows us to build incredibly detailed, realistic models – inside and out – without the risk of parts overlapping or gaps appearing where they shouldn't. It keeps the construction clean, organized, and fast.

Yes, it takes a moment to set up this base blend shape, but once it’s ready, we can fly through client requests. More importantly, it allows us to do things that standard painted-on textures simply can't handle.

Because our planks are actual 3D objects and not just images, we can easily create a wrecked version of the boat. We can pull planks off, create realistic gaps, or show splintered wood for a specific scene. We can bend individual boards or warp the wood to make it look weathered and old. If a client says, "Make the boat look like it crashed into a reef," we don't have to rebuild everything. We just tweak the existing 3D planks.

Considering a real-world application, this method's true benefit emerges with scale. While it might seem excessive for modeling a single boat, this approach significantly accelerates the workflow, particularly when handling feedback and revisions. For an example of speed, the creation of the above boat, including the creation of the base assets (SubD modelling and UV unwraps), texturing and creating the distressed version, took just under a day and a half of work.

Proven at scale

The UV blendshape modeling methods used for this blog are exemplified in the Eyes of Wakanda series. This approach was utilized to create all the ships and the hero asset, the Trojan Horse, featured in the show. By employing a base library of assets, it was possible to achieve a final quality result, populate many builds, allow for a huge variety of ships, and facilitate fast-paced turnaround on feedback.

More advanced use cases

We've explored a straightforward application of the toolset, resulting in a logical outcome. Since using its capabilities to place planks on a boat is quite basic, we will now examine more complex implementations to discover more innovative build approaches.

Let's say we have a client who has delivered us a tileable Celtic knotwork pattern, which was designed in-house by them and cannot be changed in look, topology or vertex order.

We have a very short turnaround to apply this pattern to a helical spiral without distorting it. This could be done with a twist deformer, but when the same client then asks us to apply the pattern to buildings, shields, walls or any other multitude of shapes, the twist is no longer enough. Following the same steps, we can quickly deliver. 

For another real-world example, this technique was used to create the tiling and intricate detailing on the buildings created for Asgard in the Thor: Ragnarok film, showcasing how powerful this approach to creation can be. 

Final use case examples

This technique is a massive timesaver when working with simulated clothing (like a shirt that ripples and folds naturally). It allows us to take a "finished" piece of clothing and completely change what it's made of, while keeping all the realistic folds and movement intact.

Imagine a client hands us a shirt in Marvelous Designer that has already been simulated to match a character, and with that delivery, we’re given a tiling chainmail mesh. Now, they ask us to change that fabric shirt into a chainmail top, using the chainmail geo, in a way that can then be simulated.

Trying to manually place thousands of metal rings onto a moving, folding shirt would be a nightmare. Even if you managed it, dealing with notes would only compound the difficulty.

But, if we use our blendshape trick, we lay the chainmail pattern onto the flat version of the shirt, and it automatically wraps itself into the 3D shape, following every fold and ripple perfectly. When the client has a note or a base model change, we just update the blend and deal with the note.

A similar result can be achieved using a vertex transfer via UV maps, but I prefer the blendshape method. It acts like a dimmer switch instead of just an "on/off" toggle. This gives us precise control –  for example, we can dial in a 50% blend to influence the shape. With this workflow, it’s all very simple. 

This technique ������’t just for big construction projects like boats; it’s also perfect for quick customizations. If we want to show a potential client something impressive or create branded merchandise, we can use this exact same process to add logos to clothing. It is an especially powerful approach when we have received a pre-simulated mesh, with unusable base topology (such as that Marvellous Designer produces), where we can quickly rebuild it into a working asset.

We can then take a company logo, like the �㽶��Ƶ logo and "flow" it onto the hoodie. Because it is "wrapped" using the blendshape, it doesn't just look like a flat sticker. It follows the folds and wrinkles of the fabric perfectly, giving us a real-world result. We can swap different logos or designs in and out in seconds, allowing us to show a client ten different branded options in the time it used to take to make one.

As you can see, by shifting our mindset and adopting these smart techniques, we can build very complex designs without getting stuck in slow, old-school construction methods. One artist can work on the blendshapes, while another prepares textures, another the patterns to deform onto the final shape, etc. The workflow can be run in parallel, further increasing the turnaround speed of any potential tasks.  

We can create high-end, detailed work in a fraction of the time without the need for long and tedious processes. Because the system is flexible, we can handle client feedback and design changes rapidly rather than starting from scratch. By letting the blendshape handle the hard work of bending and wrapping shapes, artists can spend more time on the details that make the project look amazing.

We aren't just working harder; we’re working smarter. This workflow ensures we stay on schedule, stay within budget, and deliver a final product that looks like it belongs on the big screen. Because we have our base patterns laid out, we can also swap in low-res, real-time-ready assets, run the wrap, and we are good to go!

This workflow is a game-changer; I highly recommend testing it out and adding it to your creative toolkit.

You can find the script here:

Daniel Murdolo is a Lead Engineer at �㽶��Ƶ. He comes firmly from the creative side of the programming fence, following a degree in game design and a varied career as a Graphic Designer. His experience ranges from designing print and television promos for a shopping channel to building mobile and web games for an advertising agency. In recent years, Dan has specialized in social AR experiences for beauty, tech, and apparel brands – including work on an award-winning integrated marketing campaign featuring an AR-enabled artist collaboration t-shirt. Currently at �㽶��Ƶ, Dan is using all these knowledge-building tools to enable others to create shared physical and digital experiences.

Tell us about your role at �㽶��Ƶ! 

I’m the Lead Engineer for the UK Web team, working on developing the OKO Web client, which is the first touchpoint and introduction for many of our users. I also go where needed, contributing to a range of studio and internal projects and assisting other OKO teams.

What three skills are essential for anyone in your role?

Flexibility would be at the top of my list, especially as you gain seniority – you never know what you’re going to be asked to write code for next.

Creativity is next. It’s a key part of programming and is hugely important for developers, even if it doesn’t seem obvious. Lastly, I would say remember to give yourself a break – programming can be hard, we all have off days, and self-doubt is ever-present. Taking time to work on things, and if you don’t get it right the first time, coming back to it later, and giving yourself that space, is as important as knowing programming languages.

How do you want to leave a mark on the world – personally or professionally?

This is a tough question for me. I think I’m still trying to find out what exactly it is, even after all this time! I think I’d like to be part of creating something that brings people joy – whether that’s through a website, game, show, experience, or the like. I know what I like to watch and play, and how those things make me feel, and if I could give someone else that same feeling through something I helped create, it would make me happy.

Outside of the digital space, contributing to a community I’m part of and seeing it grow – though I think I’m still looking for that opportunity too.

What are some things you’ve had to unlearn?

I’ve definitely had to unlearn the assumption that people think the same way I do about tasks or requests, both personally and professionally. Because you think a certain thing can be done a certain way, it doesn’t mean everyone else is thinking that too, and if they don’t do what you’re expecting, it doesn’t mean you’re any more correct than they are. 

This shift in thinking has led me to become a better communicator, which for me didn’t come naturally, and I think can be exacerbated by being a software developer if you’re working alone a lot. Though I’d say I am still definitely working on it. 

How do you approach challenges and setbacks?

Trying to distance myself from the challenge or setback and look at it rationally is what I try to do – with various levels of success depending on the situation.

Giving yourself the space to look at the problem and consider it from different angles can help you notice things you may not usually spot or think of. There is definitely a skill to being able to do this while working under tight time limits, but I believe the more you practice, the better you get at being able to do this quickly.

If you didn’t have to sleep, what would you do with the extra time?

This is probably a really cliché answer, but I’d spend more time creating and making things! I have a long list of side projects – from building games and websites, to creating pixel art and electrical projects – that I often start but don’t finish (or have yet to start at all). I figure at this point, I’m saving this growing list for my retirement.

How would your friends describe you?

I’d like to think my friends would say I am dependable and will follow through with things I’ve said I’d do. I’m usually the quiet one in my social groups, but if you need me for something, I’ll do my best to be there. 

Where is one place you want to travel to in the world, and why? 

I’ve been quite lucky in life and have been able to travel to a few places I’ve had on my bucket list, but a place I’ve yet to get to is Bolivia. 

My step-nonna (grandmother) was from there, and I never got to learn much about it from her. So, being able to visit and experience where a small part of my family unit is from is up there on my list.

I imagine it will be like nowhere I have ever been before, and exploring places like that is important for me to understand how we all fit into this big, varied world.

What is something you think everyone should do at least once in their lives?

Learn another language as fluently as you can, though I am still working on it myself! I have absorbed many small parts of different languages from growing up and from my travels. But I have yet to get to the point of fluency in a language. 

I believe you can experience visiting other places in the world in such a different way if you can communicate in the native language, and it opens up so many other opportunities generally.

Tell us something that would surprise us

In a past job, I spent the night in a shutdown jail in Australia as part of a crew filming a pilot documentary episode about a Paranormal Investigators group – and joined in the attempt to communicate with the ghosts of prisoners. The standout memory was at 3:30 AM, sitting in a jail cell in a circle with a “spirit box” in the middle, trying to talk to the prisoner who died in that cell. Do I believe in ghosts after this? I’m still undecided. But it was an equally spooky and fun night spent in a 178-year-old jail!

Welcome back, vault dwellers!

In celebration of the release of season two of Fallout, �㽶��Ƶ and Amazon MGM �㽶��Ƶs launched the World of Fallout experience. For the first time, we gave fans everywhere the chance to go behind the scenes and explore the series from the comfort of their own vault in a brand new way – exploring scenes built with the same VFX assets seen in the show.  

While the initial release showcased Vault 33 and the Farm, we’ve since expanded the experience with two further iconic scenes: the mighty Caswennan airship used by the Brotherhood of Steel and Mr House’s penthouse lair. As the long wait for season three begins, we’re diving deeper into how we built these new environments – taking high-end virtual production assets and distilling them down to run in a humble web browser.

The jump from a TV screen to a browser is more than just a change in size; it’s a total shift in how the world is "drawn”. The difficulty essentially comes down to three things:

• Size: Film-quality assets are massive. While a TV studio can handle terabytes of data, a web experience needs to be "light" enough to load over a standard internet connection without a long waiting screen.

• Memory: Browsers have strict limits on how much memory they can use. We have to compress high-resolution textures into smaller, smarter packages so the browser doesn't crash or stutter.

• The "one-size-fits-all" problem: Unlike a TV show where the image is fixed, a web experience has to look great and run smoothly on everything from a high-end gaming PC to a three-year-old smartphone.

To meet these constraints, our first step across the entire project was a massive geometry pass. We meticulously rebuilt and optimized every model to drastically reduce their triangle counts, stripping away unnecessary polygons while fighting to preserve the iconic silhouette and detail fans expect from the Fallout universe.

Caswennan: scaling the skies 

The original Caswennan models used in the show were built using a complex material-layering system in Unreal Engine – too “heavy” for a browser. To ensure the scene ran smoothly across all devices, we made the following adjustments:  

• Material overhaul: Beyond standard geometry fixes, we reworked the textures and UVs. This allowed us to convert the surfaces from a layered setup into a streamlined web-ready format.

• Texturing strategy: We maximized efficiency using a combination of repeating patterns (tileable textures) and UV mapping various assets to different sections of shared texture sheets (trimsheets). 

• Performance tuning: To maintain scene performance, we removed or simplified objects located far from where the player actually travels.

However, the goal was to strip away the data, not the personality. To bring the airship to life, we knew we couldn’t just have a static model suspended in the air – after all, we’re supposed to be soaring over the Wasteland. So we injected a sense of motion through a range of visual effects, from waving flags and wind socks to burning thruster engines and drifting clouds. 

The secret to "lightweight" motion

Because the experience uses Waypoint navigation (fixed camera spots), we found a clever shortcut: we "baked" animations into video textures and displayed them on flat planes. To the user, it looks like a complex 3D simulation; to the browser, it’s as easy to play as a standard video.

The moving flags, engine and clouds, as well as the dynamic ambient audio, make it feel more life-like.

To help sell the effect, these aren’t just flat images – they use full PBR (Physically Based Rendering). This ensures that the scene's lighting hits the moving flags exactly as it would a 3D object. To achieve this, we baked each component of the material into a texture atlas video:

• Base Color: The iconic colors of the Brotherhood of Steel.

• Normals: Dictates how light bounces off the surface to create depth.

• Alpha: This "cuts" the image so the edges of the flag are see-through.

• ORM (Occlusion, Roughness, Metalness): A three-in-one map where the Red channel handles shadows, Green handles shine, and Blue defines the metallic surfaces.

Collaboration

Before we dive into our next scene, it’s worth noting how we managed this whole effort. World of Fallout is built using our Connected Spaces Platform, which enables unprecedented collaboration. Artists can work hand in hand with engineers, content owners, and each other in real-time across the globe – whether they’re working within the web, Unreal Engine, or Unity.

Real-time collaboration: The OKO web editor (right) shows updates simultaneously in the World of Fallout application (left).

Here, you can see that changes to the scene made in the web editor are updated in real-time in the application. Multiple users can edit the scene simultaneously, leave annotations for feedback and approve changes all without leaving their browser. This is possible not just from the editor but in the Unreal Engine via the OKO Plugin. This allowed our teams across the globe in the US, France, and the UK to work together in real-time. A life-saver for this project!

The penthouse

For the final scene, we decided to go all out. Working with a single room presented an opportunity for us to push our technology further than we had before. The primary hurdle here was the "one-size-fits-all" problem. In traditional VFX, beautiful reflections and glass refractions use a technique called “Path Tracing” to accurately simulate light bouncing off surfaces, something that is extremely slow to render. In a web browser, they have to be calculated in milliseconds, whether the user is on a high-end desktop or a three-year-old smartphone.

Mastering reflections

The standout feature of this scene was the shiny marble floor. In real-time graphics, reflections are among the most rewarding effects to achieve. To rise to the challenge without crashing mobile browsers, we introduced a new enhancement to the engine: Reflection Probe Volumes.

Reflection Probes enable us to blend multiple volumes (also known as per-pixel blending), resulting in much higher fidelity reflections that integrate naturally into the scene. By layering these probes within one another, we can concentrate reflection detail where the player looks most, while ensuring the rest of the environment remains visually cohesive, regardless of the hardware's power.

The illusion of refraction 

The other standout feature in the penthouse is the secret room, in particular, the bubbles within the cold fusion device. If you look closely, you can see how the glass distorts the bubbles around the edge. In a standard game engine, this would require complex "refraction shaders" that often fail on lower-end mobile devices.

So, how did we make it work for everyone? It's video textures all the way down. The distortion is “baked” into the video in such a way that it perfectly lines up with the edges of the 3D glass, giving the appearance of real-time refraction. It’s a classic "movie magic" trick – delivering a high-end visual result with almost zero impact on the device's memory. It looks like a complex simulation, but it runs as smoothly as a standard YouTube video.

It's what’s underneath that counts

Creating a cutting-edge, real-time 3D experience requires a solid foundation. To reimagine the world of Fallout for the web, the engineering team at �㽶��Ƶ built a custom renderer on top of PlayCanvas, optimized for this experience. For more details on this, read our article Building Beautiful Worlds Across Platforms with Playcanvas and the �㽶��Ƶ Web Renderer.

Engine enhancements such as LODs (Level of Detail) and a custom optimization tool we built called gltf-tools (based on the excellent ) ensure that we only render what's needed. By combining the precision of film-grade VFX with the custom-built efficiency of the �㽶��Ƶ web renderer, we've successfully distilled a cutting-edge real-time 3D experience for the humblest of platforms.

This deep dive into the Caswennan and Penthouse is more than just a showcase of technical tricks; it’s a demonstration of our commitment to pushing the boundaries of what is possible in a web browser. 

As we continue to develop the Connected Spaces Platform and look to the future, we remain dedicated to delivering high-fidelity, shared experiences that truly allow fans to step inside the worlds they love.

Go experience the World of Fallout right now at

Watch Fallout, Season Two on Amazon Prime Video

Amy studied Multimedia Design at university, a degree that allowed her to wear many different hats on projects, ranging from Producer and Designer to Engineer and Tester. Hungry for real-world experience, she volunteered with the West Midlands Fire Service within their Learning & Development department. There, she supported content creation and the prototyping of a learning tool for safety procedures and badges.

Her professional journey then took her to a Communications & Event Agency. While she joined the Digital team as a Project Manager, she transitioned into Quality Assurance ten years ago and has never looked back. Seeking a fresh challenge beyond the traditional, Amy joined �㽶��Ƶ nearly four years ago.

Tell us about your role at �㽶��Ƶ! 

As a QA Manager, I lead and manage our UK QA team, overseeing all QA activities and ensuring projects hit their quality goals. It’s about anticipating and identifying problems, recommending effective solutions, strategically adapting the QA approach to different project needs, and collaborating with other disciplines, clients, and end users to deliver on requirements. 

What made you decide to pursue a career in this field?

Being inquisitive, not being satisfied with following the ‘happy path’. Striving to make solutions better for everyone.

What has been your proudest/standout moment while working at �㽶��Ƶ?

Being part of the team that launched OKO and the Connected Spaces Platform (CSP) – our products that connect physical and digital spaces. Aside from this, I take great pride in seeing my team succeed – getting to play a small part in supporting their successes.

What’s your favourite thing to do when you’re not working? 

All the sports! Reading, watching, and playing them! You’ll regularly find me on some sort of green surface, be that playing or coaching Hockey (Field Hockey, definitely not the kind on ice!), or playing Crown Green Bowls (there is nothing more inspiring or humbling than losing to an 80-year-old woman).

If you could have any other job in the world, what would it be?

It may seem random, but it would be a Chef. There are a lot of qualities that align with my actual role and discipline. Plus, I love trying new ingredients and cuisines.

Tell us something that would surprise us!

I’m extremely competitive, and I play to win! Also, in my youth, I played football with the most recent winner of Strictly Come Dancing.

If you could wake up in the body of another person (just for one day) who would it be and why?

Fred Again.. Mainly because so far I have been unsuccessful in getting to attend one of his gigs. So to experience the world as the maestro himself would be incredible. Otherwise, it would be my dog Edie Pug, as she lives like a queen!

How do you want to leave a mark on the world – personally or professionally?

I want my mark on the world to be defined by the way I listen. We live in a chaotic era where being heard can be rare, so I strive to be a constant, supportive presence for those around me. Whether I’m leading a team or supporting a friend, I hope to leave behind a legacy of empathy – ensuring that no one in my circle ever feels invisible or without an advocate.

Do you have a go-to karaoke song?

Karaoke is my worst nightmare; I would never subject anyone to hearing me sing.

Fog is both a technical and artistic challenge – it drives mood and depth, but if you get it wrong, the entire scene can collapse visually.

In this article, I'll discuss how fog enables you to create moody, mysterious environments with depth, movement, and character. I’m using the , to illustrate my points.

Let’s dive into four types of fog.

Exponential height fog

The first step in creating a moody foggy looking scene is to play with the settings in the environment mixer panels (Window>Env. Light Mixer). Combined, these tools are very powerful and can get you far in terms of setting the mood for the lighting on your project.

Here you can see my scene with and without exponential height fog turned on.

The first thing you need to do is set up the scene, complete with everything from your environment light mixer, as shown above. Then, enable volumetric fog in the exponential height fog settings.

You can play with the fog height falloff and the density to control how thick and intense your fog is. I needed my fog to cover the mountain in the background, so I had to lower the fog height falloff value. 

The next important thing to consider is the direction of your light. God rays are essentially light that filters through the fog. Since it’s occluded in certain areas, it forms rays.

I tweaked the position of the trees to get a nice result. I also created a mask to add more detail to the god rays. The mask is just a simple 3D surface with a special texture applied to it. This noise texture controls transparency, going straight into the opacity channel. I only added a multiply setting to control how much light goes through, and a soft mask to ensure there are no hard edges. The size of the noise is adjustable.

Another setting you can look into is the Volumetric Scattering Intensity on your directional light, which makes your god rays stronger.

To add some movement to my scene, I did the following three things:

First, I added a moving distortion to my mask for the god rays. It’s subtle, but I think it adds a lot!

Next, I added a god ray card that’s included with the engine.

When you go into the engine folder and look for a beam, you’ll find there are a lot of things that just come with the engine. I’ve circled the items I copied over to my project folder in yellow, and circled the one I selected for the final scene in green.

Once you have dragged the god ray into the scene, go to the details panel to find the master material. I created a material instance, which makes it a lot easier to tweak the values. It’s important to tweak the color and the opacity based on your scene.

When you add the texture overlay, you can set a panning speed to add some movement. Again, it’s a subtle change, but you can feel how it adds to the scene.

Lastly, I added a dust spec particle effect. Now I’m no VFX expert, but by and customizing it to my needs, I had it working in 10 minutes – and I think it really helps sell the sense of movement!

Fog cards

The second technique I’m going to cover is fog cards. They let you control exactly which shapes get more definition and what noise blends into the background.

For my scene, I started with a really simple fog card, which was a soft circle that blended into the environment.

To achieve this, you need to set the material’s blend mode to “translucent” and check “two-sided”.

When you make an instance, you can adjust the opacity, color, and emissive value. Since this is just a card with the image of a cloud on it, the illusion would break if the camera went through it. Therefore, we make it fade away when the camera comes close.

Even with this simple functionality, we can use the cards strategically to shape our scene and reduce contrast behind key elements, making the silhouette stand out more. Here, I’ve placed a few simple fog cards to help define the composition; these cards add depth and draw attention to the right forms. It's still a work in progress, but the overall vision is coming together!

While there's already some atmospheric height fog in the scene, these cards give us even more control. Exponential height fog affects the entire scene, but the fog cards are localized and great for fine-tuning.

Next, let’s add some detail to our fog cards. To do this, I duplicate the same noise texture with a different scale. I then blend them together and add a panner to both of them. You can set the speed to different values to give the illusion of the fog changing shape. I then multiply the created moving noise by the opacity. I added a lerp, so we can interpolate between just having the mist or the moving noise, because we want control of how much noise we get. I also added the object position to the texture coordinates, that way, every fog card looks unique, based on its position. We don’t want all our fog cards to have identical noise.

In this image, you can see the updated lighting, improved water, and there’s also a little more foliage. But most importantly, our fog now has texture and movement. It’s a bit intense at the moment, but this is to make it easier to see how it behaves.

The fog has detail and movement now, but is still lacking depth.

To add depth, first change the lighting mode of your material to “surface translucency volume”. Note that this makes the material a bit heavier to render.

Export the noise texture and use Substance Designer (or use a similar package ) to convert it into a normal map.

Then copy the same logic for the panner of your noise texture and apply it to the normal maps, though there is a key difference to note!

Normal maps are controlled by the Normal Flatten node; if you want to make it stronger or weaker, you need to use Blend Angle Corrected Normals to blend the normals instead of a lerp.

Other than that, the logic is very similar.

Here, you can see how the fog now updates when you move a light around, as though it’s an actual fog cloud with volume instead of a flat plane.

Here’s the scene with our new fog. The chimneys are lit and the dust effect from the previous chapter has been reused. I’ve also added the fire to highlight how the fog responds to that type of lighting.

For the best results, it’s important to create a material instance and tweak the values of your parameters to the look you want.

I personally like to arrange them into groups, so it doesn’t become too overwhelming.

To bring that point home, here’s what the scene looks like when you rotate the directional light. You can see that the fog works well in different lighting scenarios. This is especially handy when you need a scene to work for multiple times of day!

Fog volumes

Fog volumes can be super handy when you want to feel up close, surrounded by fog, but only in one local area – they allow you to control the fog in a given volume. Within that volume, you can give the fog a cool shape, use noise, make it move, control the density, and even make it cling to surfaces.

To get the simplest fog volume, create a material, then set the material domain to “volume” and the blend mode to "additive". To start, I’ve just put a color for the Albedo and a value for the Extinction to see what it looks like initially. Note that you still need an exponential height fog in your scene with volumetric fog “on”, for the fog to work.

Apply the instance of your material to a cube, then play with the values. You can get a decent result even with this simple setup!

Here it’s used to add some low-hanging fog over the water. Note how the volumetric fog becomes lit by the golden sunlight and gets less visible where the sun is occluded by shadow. It adds depth to the cat tail grass in a way that would be impossible to achieve with fog cards. To get a similar effect with an overall exponential height fog, you would have to flood your entire scene with heavy fog.

But this ������’t ideal yet. The transition when you go under the fog level is too harsh, and there is no definition or movement to the fog at all. So let’s have a look at how we can make it better.

The following setup will round and distort the boundaries of the volume towards spherical, pushing it away from cubic, when the fog edge roundness is adjusted. Here’s how the shader graph works:

•  The “local position” node gives you the position of every vertex, so if you subtract the minimum position of an object, it gives you the position of every vertex relative to the 0 point. 

• Divide that by the extent, and now you have a gradient from 0 to 1 in every direction (X=R, Y=G, Z=B).

•  If we calculate the distance of those values to 0.5, we get a gradient radiating from the center. 

• We can add a parameter that determines the scale of the gradient. 

• We have to one minus it because the center of the gradient will be 0. And we want to create a mask that is 1 in the center and 0 at the edge, so we get the full amount of fog at the center.

Next, we want to introduce some noise. Since this is a volume, we need a volume texture. Luckily, you can get this Perken Worley noise volume texture from the in-engine content!

To add it to the shader graph, you need to look for a “SparseVolumeTextureSampleParameter” and apply your volume texture. Similar to the fog planes, where we add logic to control the size or tiling of the noise; we add a panner here, now with three dimensions of speed. We use the power node to control the contrast and the multiply node for the intensity.

Now when you play with the values, you can get a fog that moves and has some visual variation.

Next, just like with the fog cards, we’re going to combine two sizes of noise together and give them different speeds.

While tweaking the material, I added an extra multiply before the speed and before the size, so I could tweak them simultaneously. And this is the result! I put a spotlight in the house and turned on the “cast volumetric shadows” option.

Note, don’t forget to turn off the collision of your cube!

You can change the voxel size with these commands: r.VolumetricFog.GridPixelsize 16

This will set it to 16, but playing with the value is the whole point!

If you use “r.VolumetricFog.GridPixelsize ?”, it will output the current pixel size in the Output log. When you change the gridPixels this way, it will make the fog look smoother, but it increases the atmospheric fog resolution for the entire scene, which very quickly adds up and affects the scene optimisation.

Next, we’ll add some colors to the fog. We can get a different color for the light and shadow of the fog by using a lerp between two colors, which is controlled by the output of the noise we generated.

To add a gradient to the noise, we can use the B value of our gradient to make our shape. We multiply it by a value that will determine the spread of our gradient, and then use a smoothstep node to create a nice interpolation. 2 parameters, between 0 and 1, will determine where the gradient will begin and end.

In this gif, you can see how I’ve used the new gradient functionality to make the top of the fog orange and the bottom blue. It makes it look like the bottom part of the mist is cool from the water, and the top part is warmed by the sun.

You can also add an emissive value – just multiply it by the base color. This is mainly for when the fog is in the shadow or in the dark, as it can disappear otherwise. Since I quite like the volumetric shadows to do their thing, I usually leave it at 0, but it can be handy in certain situations.

The next step is to add in another noise. This time I’ve used this “VT CurlNoise” that you can also find in the Engine files. It reuses the panning and tiling functionality of the other noises. We also reuse the absolute world position and the tiling of the other noises, that way they will match. What this does is scramble the noise based on the curl noise, giving us a more interesting, more defined noise.

Using this technique, you can see that the fire is lighting up our noisy fog; the top of the fog has a pinkish hue, which turns to purple blended with our blue night scene, giving a pretty magical falloff.

The next thing to add is the distance field. You can see that I’ve added this bit of logic at the bottom left. We want to control the distance and the hardness of the edge, and then add that to the current opacity. There’s a switch to control if we want the distance field or not.

When we turn that switch on, we can see our fog wrap around the meshes of our scene, which looks very natural.

As with the fog planes, it’s super important at every step to tweak the values of your material instance to achieve the look you want.

Here I went for a cool, crisp, misty morning.

VDBs

VDB is a file format that stores volumetric data, like smoke, fire and clouds. It can store different parameters, like density and temperature. It stores the data in Voxels, which control the detail level of the cloud. If we want more detail, we need more voxel density, which increases the size of the file.

You can create VDB files in software like Houdini and Embergen, but you can also download free samples from the , which is what I’ve used in this case.

To import them, drag them into your folder structure and select the parameter you want to include. I’ve just included density and nothing else in this case. The engine will convert them into sparse volumes.

To get it in your scene, you need to add a heterogeneous volume.

It expects a material rather than a VDB. To get this material, go under your engine files again, and find your “SparseVolumeMaterial”.

When you create an instance, you can input the sparse volumes you have imported into the sparse volume texture slot

When I first looked at it in the scene, it looked a bit too intense! Lowering the density gave me a pretty nice result, so make sure to tweak the settings to suit your needs.

In this case, I used a static VDB texture, but you can use animated VDBs to get very detailed animated fog. The drawback is that it can be very non-performant, so use it at your own discretion and assess whether it’s worth it on a per-project basis.

To conclude…

These are the four main ways to achieve fog in Unreal 5. All of them can be used to create beautiful, moody scenes. Here you can see what a difference toggling the fog on and off has.

Adding these different layers of fog can really help take your scene from good to amazing, so I highly recommend experimenting.

Hopefully this article helped to demystify fog for you!

Ben has spent nearly seven years in recruitment, beginning in the agency world before transitioning in-house to focus on building lasting teams he could also be part of. At �㽶��Ƶ, he’s been involved in hiring across programming, art, design, and support functions in the UK, helping bring the talented people behind our amazing projects on board. Working in the immersive industry means thinking outside the box to find the right talent – a challenge Ben thrives on!

Tell us about your role at �㽶��Ƶ.

Day to day, I work with hiring managers and senior leadership to figure out the team and the wider company's needs, then go out and find the right people to join us. Right from the start, my focus is on communication and building a real connection with candidates. It's about genuinely getting to know them and ensuring they feel informed and prepared every step of the way. I manage everything from that first conversation to keeping interviews on track, and I contribute to hiring strategy when needed. Because the roles I cover are so varied, there’s always something different going on – something I really enjoy. The best part for me is seeing someone settle in and do well here. Aside from day-to-day recruiting, we’re always looking for ways to improve our processes and keep the candidate experience top-tier, so there are always side projects on the go, too.

How did you get into recruitment in this industry? 

Before �㽶��Ƶ, I’d mostly been recruiting in financial services, so stepping into creative tech was definitely a learning curve. I came across the role and checked out the work �㽶��Ƶ was doing – it immediately caught my eye. Seeing the projects on the website made me think it would be incredible to be part of that. Even though I hadn’t hired in this industry before, I’d spent years recruiting for niche and technical roles, so I knew I could bring those skills to the table and learn the rest along the way. Looking back over the last three years, the projects we’ve worked on have been even more impressive than I imagined.

What do you look for in applicants? What makes someone perfect for �㽶��Ƶ? 

In my experience, the people who thrive at �㽶��Ƶ aren’t just skilled in the tools of their trade – whether that’s a game engine, DCC package, or something entirely different. What really makes someone stand out is their mindset. Our work is often experimental and pushes the boundaries of what’s possible, so we look for people who love exploring new ideas, figuring things out, and learning by doing. Collaboration is a huge part of what makes our projects work, so being someone who shares knowledge freely and builds on others’ ideas is key. If you’re naturally curious, creative, and enjoy solving tricky problems together, you’ll fit right in here.

What three skills do you think are essential for anyone in a recruitment role?

If I had to pick three skills, I’d say communication, empathy, and resilience. Communication is key for connecting with candidates and hiring managers, empathy helps you understand people’s motivations and experiences, and resilience is essential for navigating the ups and downs that come with recruiting.

If you had unlimited resources and funding, what project or initiative would you launch? Doesn’t have to be work-related! 

If I had unlimited resources, I’d put them into changing the way English speakers (especially us Brits) think about learning other languages. We rely so much on everyone speaking English, but you miss out on so much culture when you’re unilingual. I only changed my mindset in my mid-20s when I started learning French, and I can say firsthand how tough it is to learn as an adult. But it’s been so worth it. I’ve had conversations with strangers while travelling in French-speaking countries that I never would’ve had otherwise. If we encouraged that mindset earlier in life, people would get so much more out of the world.

Tell us something that would surprise us!

I used to compete as a national-level springboard and platform diver when I was younger. It feels like a lifetime ago now, but the skills still come in handy whenever I find a cliff on holiday!

What’s your tactic for surviving a zombie apocalypse?

Honestly, I think everyone’s outlook in a zombie apocalypse would be pretty bleak. I’d probably just let the first one bite me and hope my experience was more like Warm Bodies, where you keep your consciousness and can carry on living.

One place you want to travel to in the world?

I’d love to visit the Norwegian fjords. The idea of staying in a little log cabin with a sauna, surrounded by the water and mountains, sounds pretty perfect to me.

What’s your favourite thing to do when you’re not working? 

Long walks with friends where we frequently stop in pubs along the way. I think it has a name?

A look at the past

The year is 1983. The age of mass interconnectivity is dawning. Simple Mail Transfer Protocol (SMTP) debuts, and the concept of electronic mail is born. Suddenly, you don’t need a pen, paper, and stamp: just an email address and a computer. Yet users still crave real-time, text-based communication – something more immediate than email. 

This desire drives the creation of the first mass-adopted text-based communication technology: Internet Relay Chat (IRC), which launches just five years later in 1988. For the first time, people can communicate with each other through text on a screen. The ability to chat with anyone, anywhere, in real-time is a historical first.

Technological innovation has always outpaced expectations, and 1993 is no different. CERN releases the World Wide Web source code to the public, and there are just 500 known web servers. By the end of 1994, just two years later, an estimated 10 million people are using the Web, and there are 10,000 servers. By 2005, just over a decade later, one billion users are surfing the World Wide Web and every five years after that, another billion people add to that count.

The Information Age is here – and it’s here to stay.

A look at the present

Today, consumers and businesses alike are spoiled for choice when it comes to communication: smartphones are always within reach, laptops allow people to work on the move, and even flights now offer WiFi, some as standard. Communication has never been so readily available, and yet the way we use it has, more or less, remained the same: text, images, and video on a flat, two-dimensional screen.

We’ve all mastered communication in 2D, but the irony is that even when we’re creating spatial content in 3D, we’re forced to compress it into a 2D screen format.

If you’re a 3D artist, you’ll have experienced this frustrating workflow… You’ve just created some complex 3D content, you start recording a 2D video, moving around the content to showcase it at different angles, and then you shoot the video over to the team to take a look. The team provides feedback, be it changes to the content itself or its position within the world, and after a clumsy, hard-to-follow thread of messages, you fine-tune your creation. Rinse and repeat.

Even as our technology advances into virtual, augmented, and mixed realities, we still rely on those same 2D tools to communicate within them. At �㽶��Ƶ, through OKO – our suite of interoperable apps built for creating and running cross-reality environments – we saw a unique opportunity to solve a problem that has historically been silently problematic: How can we provide people with the means to effectively and efficiently communicate with others in a 3D context?

Laying the foundations

In 2022, we realised that we had already started encountering these sorts of inefficiencies when communicating in the context of OKO. From the valuable feedback we’d gathered from third parties, our users, and our own internal teams, we concluded that OKO needed a built-in communication tool to enable better collaboration. We saw it as an opportunity to solve some of these unique and historic challenges for OKO and, by extension, our users.

Despite that, we still needed some groundwork to build on: a simple user interface we could extend while ensuring we weren’t just offering an embedded version of the same tools already on the market.

Comments

Our first implementation of Comments was a proof-of-concept, released in 2023. It wasn’t well-suited to our existing infrastructure, nor was it designed to be – this was new ground! The feedback we gathered, however, was the most important thing. It’s impossible to present the functionality of a feature via text for the purposes of gathering feedback, and so our first prototype served well.

Initially, Comments were simple. They consisted of a title, a body, a timestamp, and the author, as well as offering a button to teleport a user to the position from which the Comment was authored. For us, this was a good start as it served as a prototype we could iterate over and gather feedback from. It was, however, missing something key: it didn’t foster the collaborative energy we envision within OKO, and, ultimately, it was too similar to the existing communication tools on the market.

And so, in 2024, having gathered all of the feedback from version 1.0, we decided it was time to begin the next iteration of Comments. I won’t dive into the feedback in detail here (I’ve been told there’s a limit to how much people will read nowadays), still it demonstrated some key areas to focus on: design, reliability, functionality, and responsiveness.   

And so we focused our efforts on answering a question we’ve already asked in this article: How can we effectively and efficiently communicate and collaborate with others in a 3D context, and more importantly, how can we make that a great experience?

Initially, we considered adding the ability to add screenshots, images, or videos to give readers of the Comment some extra context to understand them. But, after consideration, this was too similar to existing tools that already offer built-in conversations with file upload functionality.

And, in the context of OKO, it didn’t really make much sense.

When large groups collaborate, it typically results in large-scale changes with added complexity. But that led to problems as communication relies on text, even when the ability to upload images or videos is available. What if someone comes along and moves the object? What if it no longer even exists? How do you annotate something, like you would on a screenshot, that will very likely move around or change in appearance?

Annotations

After Comments, we were finally at a point where we could tackle the hard part: 3D spatialised communication. After considering screenshots and/or videos, we decided to do something quite radical: a second, static camera with built-in drawing tools in the UI. Painting on a camera frame, if you will. Annotations was a fitting name for the feature.

We did face one key question, though: if a user adds an Annotation and the spatial context changes, how do they know what it was referencing in the first place? To solve that, we decided to fall back to something we’ve discussed a lot already – screenshots. The thumbnail for any Annotation is a screenshot of the spatial context the author saw at the time of authoring the Annotation. We’re not so anti-2D after all!

Our main problem was one of our own (and intentional) making. OKO supports almost any device – from smartphones to headsets to desktop computers and laptops. We had to account for infinite screen resolutions, aspect ratios, and field-of-views (FOVs). Handheld devices, such as smartphones, vary in resolution and aspect ratio and can’t be changed. In contrast, desktop devices are totally configurable and dependent on external hardware such as monitors. In addition, the engines we build on, such as Unreal Engine 5 and Unity, offer fully customisable FOV settings, with UE5 using horizontal FOV and Unity using a vertical FOV.

All of these variables were important to control, accurately convert, and store as part of each Annotation to ensure that when one user views another user’s Annotation, they both see the same thing. As with Comments, we began the long process of testing these variables on multiple devices with different settings to see whether we could accomplish this.

We now had the data that we needed to collect and store:

1. The author’s coordinates at the time of creation

2. Their aspect ratio

3. Their vertical FOV

4. A screenshot of the spatial context at the time of authoring for comparison

Despite the complexities, this unique approach solved several problems that screenshots or videos have historically had.

Firstly, if one user is using different settings from another, then we can display an accurate representation of the 3D context of the Annotation. With videos or screenshots, things can change massively due to user-specific or device-specific settings, making it challenging to understand exactly where they were captured in a spatial context, especially in larger areas.

Secondly, if the context of the video or screenshot changes or is moved around, then they’re no longer accurately represented within a spatialised context. With Annotations, this change is immediately visible because the thumbnails show a before and the Annotation, when viewed, shows the after, leaving no room for ambiguity.

Finally, the long-discussed problem we solved: text is simply not very good at describing 3D spatial context. Annotations leave no room for ambiguity. When combined with Comments, Annotations really come into their own: this is the author, this is what they want, this is what they saw when they wrote it, this is what they drew, and this is what it looks like now.

For over five decades, we’ve relied on text, images, and videos to convey complex ideas on screens. With Annotations, we’ve challenged the status quo and developed a unique solution that moves conversations and collaboration inside of the experience rather than outside of it. This fundamentally improves the speed, efficiency, and effectiveness of spatial creation – from digital twins to virtual concerts and beyond.

As a senior member of the HR team at �㽶��Ƶ, Camille Hall provides dedicated support across all employee functions, building on three years of service with the company. As a certified SHRM-CP, she brings comprehensive expertise across all core pillars of HR and supports employees throughout the entire employee lifecycle. Her foundational expertise is built on a B.S. in Psychology and professional experience that began in the healthcare sector, allowing her to approach HR with a blend of empathy and structured support. A proud native of Minneapolis, MN, she still firmly believes the Midwest is the best place – and possibly the only place – you can truly master the art of "you betcha!".

Tell us more about your role at �㽶��Ƶ! 

My role at �㽶��Ƶ is essentially about being a dedicated partner and advocate for our employees, providing strategic HR support and structure so they can focus on creating extraordinary experiences. 

My responsibilities cover the entire employee lifecycle at �㽶��Ƶ, from first day to final sign-off. On any given day, my focus shifts rapidly: I pivot from acting as an unofficial therapist in meetings, to becoming a professional data diver working through ADP reports. I welcome new hires and then transition to being a benefits guru who swoops in to rescue employees from the confusing abyss of healthcare plans. 

What attracted you to �㽶��Ƶ?

What brought me here was the thrill of the unknown. Coming from healthcare, the immersive technology space was completely new – I had very little exposure to augmented or virtual reality. I saw �㽶��Ƶ and knew I wanted a challenge, and, honestly, I wanted to be part of something groundbreaking.

What is the biggest lesson you’ve learnt in your career?

The single biggest lesson I’ve learned is that everyone’s ‘best’ looks different. My transition from corporate healthcare, which was often very "black and white" and process-driven, to the creative world of �㽶��Ƶ drove this home. In HR, I realized you can't fit everyone into a single mold; one person’s "best" is meticulous detail, while another’s is innovative risk-taking. My job is to provide the inclusive HR framework and support that allows those diverse versions of 'best' to thrive and succeed here at �㽶��Ƶ – a level of personalized partnership that often goes beyond a traditional HR role.

What, or who, inspires you? Or what moment has had the most significant impact on your life? 

The people who inspire me most are my parents. They worked incredibly hard for years, and now they are retired and traveling the world, fully embodying the reward of that dedication. 

Their passion for seeing new places has certainly instilled my own love for travel. However, the biggest impact is the sense of adventure they passed down. It taught me the value of being open-minded, trying new things, and being willing to take necessary professional and personal leaps.

How do you think your friends would describe you?

They'd probably say I'm someone who loves to laugh and is a reliable sounding board – the person always there to listen. Beyond that, I think I’m also known as an excellent gift-giver!

What’s your favourite thing to do when you’re not working? 

I love a good nap!

Are you reading/listening to anything good right now?

I'm a massive Stephen King fan— novels are probably the best series ever written, setting a dangerously high bar for everything else. That said, I'm trying to branch out (I’ve read IT three times). I recently finished Robert McCammon's post-apocalyptic epic, Swan Song, and I'm currently working my way through Dan Simmons' historical horror novel, The Terror.

Tell us something that would surprise us!

I love ! It’s not just a fast-food joint that happens to have its own movie; it’s nostalgia. My love started early: my grandpa used to pick me up from preschool and take me straight there. Those tiny, greasy sliders became a favorite tradition that I still enjoy to this day.

What is something you think everyone should do at least once in their lives?

Hike Machu Picchu with your family and best friends.

Where is one place you want to travel to in the world?

Isle of Skye, Scotland. 

Gaussian splatting is a relatively new technique for real-time radiance field rendering, and by creating more than 1200 Gaussian splats from hundreds of thousands of photos, we’ve learned a lot! Before diving into what we’ve learned, it’s worth looking at how Gaussian splats compare with other popular approaches to 3D capture and rendering: photogrammetry, LiDAR, and NeRFs. Each technology has its strengths and trade-offs, and in practice, we often use these techniques side-by-side. Here’s a handy comparison chart:

Ten lessons

To illustrate the lessons we’ve learned, we’ll look at a case study from the Allee Willis Foundation. The foundation perpetuates the legacy and extraordinary creative vision of Hall of Fame songwriter and artist Allee Willis by fostering innovative opportunities in music education. 

We were invited to photograph the interior and exterior of the Allee Willis house, working with a team of photographers, lighters, and production staff to preserve it as a digital twin for future research and creative projects.

A note on our process
Our advice is based on our experience using DSLR, mirrorless, and sometimes phone cameras for creating Gaussian splats. While we recognize the value of creating Gaussian splats with LiDAR-enhanced cameras, VR (virtual reality) and XR (extended reality) headsets with multiple cameras and a depth sensor, and mobile AR (augmented reality) based software solutions, our focus in this guide is on achieving high-quality results with standard photography equipment. We also avoided using drones for our recent projects due to permitting complexities, though they can be useful for certain environments. For best results, we recommend using still photos over video to avoid motion blur and benefit from a higher image resolution.

The photography process
For each shoot, a simple, repeatable process is key:

• Set up: Use a single prime lens and the same sensor for a room or object for the entire shoot.

• Calibrate: Set your camera's shutter speed, aperture, ISO sensitivity, color temperature, and focus carefully, and turn off in-body image stabilization (IBIS). Avoid a shallow depth of field by using a narrower aperture to make sure that everything is in focus. Adjust settings while shooting as necessary if you are in a situation where the lighting changes during the shoot (for example: dawn or dusk).

• Shoot: Take a lot of photos, moving slowly and deliberately to ensure a high degree of overlap between shots. Do whatever it takes to avoid motion blur, even if it means using a monopod, a tripod, or a faster shutter speed.

Wherever possible, shoot in RAW format for greater flexibility, and use a polarized lens as needed to minimize unwanted reflections. Establishing a scale reference can be helpful. By following these guidelines, you'll be well-prepared to capture high-quality images for your Gaussian splat projects.

Let’s dive into what we’ve learned! 

1. Beyond the circle: alternative camera trajectories for 3D reconstruction

When you're creating a 3D model from photos using techniques like Gaussian splats or photogrammetry, the path your camera takes – the trajectory – is just as important as the photos themselves. While it's often recommended to shoot full circles or a dome around your subject, this isn't always possible. Obstacles like walls or furniture can get in the way.

The 70% overlap rule
This is the golden rule of thumb: each photo should have at least 70% overlap with the previous one. Hitting 80% is even better. A high degree of overlap is crucial for the camera pose calculations, which figures out the position and orientation of your camera for each photo. Without enough overlap, this process can fail, resulting in an unusable reconstruction.

Circles and domes
The classic circles/dome trajectory, where the camera points towards the center of a circle/dome/cylinder, works well because it naturally ensures this high overlap. There should be several circle trajectories at various heights to form a dome or cylinder. (See this article for more information on circle trajectories.)

The power of concave half-shells
Based on our experience, we've found that you can still get excellent results by adapting your approach. The key principle is to maintain a high degree of overlap between your photos.

When a full circle/dome trajectory isn't an option, our most important discovery is that concave half-shell trajectories are a fantastic alternative.

Think of it this way: instead of a full circle, you're shooting an arc. With this method, you move your camera along a curved path while it continuously points towards a single focal point. This simple approach is a powerful substitute because it mimics the natural overlap of a circular path, making the camera pose calculations far more likely to succeed. Make sure to photograph each area using several arc trajectories at various elevations, just as you would for the circles/dome trajectory.

Trajectories to avoid
Just as some trajectories help, others can make your job much harder.

• Linear trajectories, where you move in a straight line without changing your angle, can be risky. On their own, they often don't provide enough different vantage points for the camera pose calculations to work properly. If you use a linear trajectory, we recommend changing the angle of the trajectory from time to time to help the calculations succeed.

• Convex trajectories, where your camera is pointing away from the center of an arc, are an even higher risk. This type of trajectory may be tempting if there are several objects to photograph around the periphery of a large room. However, it's trickier to get 70% to 80% overlap between subsequent photos when your camera is pointed away from the center of an arc, making a successful reconstruction much more difficult. 

Key takeaway
When a circle/dome trajectory isn't possible, a well-planned concave half-shell trajectory is your best bet for capturing high-quality 3D models. It's an effective way to maintain the crucial 70%+ photo overlap needed for a successful reconstruction.

2: Mirror, mirror on the wall 

Mirrors are tricky! Flat mirrors often look great with Gaussian splats, but reflections on curved surfaces are currently a major challenge.

Gaussian splats of spaces with flat mirrors work well because the mirrors create a convincing "mirror universe," a duplicate version of the room on the other side. As long as there are no overlaps between scenes from multiple mirrors, the effect is very convincing.

The challenge of curved surfaces
In contrast, curved mirrors or highly reflective, curved surfaces do not work as well with Gaussian splats. The reason is that light is reflected at different angles from every point on the curved surface, which makes it very difficult to consistently reconstruct the "mirror universe." The Gaussian splat training process attempts to aggregate a reconstruction from these multiple angles, resulting in a distorted and messy virtual scene.

If you run into reflection artifacts on curved surfaces, you could attempt to use cross-polarized photography to reduce reflections. Alternatively, using higher-order spherical harmonics could help to enhance the visual appearance of reflections. However, even with higher-order spherical harmonics, accurately capturing complex reflections on a curved surface remains a significant hurdle. Mitigating reflection artifacts on curved surfaces is currently an active area of research. 

Key takeaway
Flat mirrored surfaces work because the reflections are consistent. Curved reflective surfaces are a struggle because their inconsistent reflections create a distorted and messy reconstruction that current technology struggles to accurately resolve. 

3: Comprehending the formless: why flat untextured surfaces are so tricky

Imagine you are in the middle of a uniformly lit white, untextured room without any discernible corners, furniture, objects, or shadows. How would you understand the shape of the room with just your eyes? Until you use your hands and feet (which are not completely unrelated to depth sensors), it’s difficult to comprehend the space. This is the same problem that computer vision algorithms face with uniformly lit, flat, untextured surfaces. The reconstruction process is essentially guessing the shape of a surface that it can't see clearly, leading to an imprecise result. 

Using image-based analysis alone, the problem with a flat, untextured, uniform surface like a white wall is that it has few discernible feature points, which the software can use to correctly reconstruct the surface. Because the software has difficulty figuring out where the wall or ceiling is, these appear as blurry, undefined "clouds" in the final reconstruction. We saw this firsthand at the Allee Willis house and were able to confirm it in controlled tests with a large white poster board.

As human beings, we can make semantic inferences, which help us to understand flat, uniformly lit, untextured surfaces like a wall or a ceiling. Some types of 3D reconstruction software are beginning to use semantic understanding of scenes to create cleaner models. We’re hoping that, in time, structure from motion and Gaussian splatting techniques could incorporate semantic understanding to help resolve flat, untextured walls and ceilings that currently appear cloudy or formless. (Although we’re not covering depth sensors in this article, LiDAR or structured light sensors, such as those on modern XR headsets, could be used to resolve the cloudy wall/ceiling issue as depth sensors have a special advantage in understanding where these flat, untextured surfaces are in space.)

There is no perfect fix, but here are some things you can do to manage this issue:

• Add visual markers: In a controlled environment, adding markers (like black dots or tape) can help. These give the software the feature points it needs to correctly resolve the surface. These visual markers could be removed later when editing the Gaussian splat. However, adding markers isn't always a practical solution for real-world scenarios.

• Vary your angles: Taking a large number of photos from a wide variety of angles can sometimes help. By capturing the surfaces from many different perspectives, you give the software more information to work with.

• Editing is an option: Sometimes, the only solution is to edit the final Gaussian splat afterwards. This allows you to manually clean up the cloudy areas (or remove temporary visual markers).

Key takeaway

Flat, untextured surfaces are currently difficult to reconstruct accurately with Gaussian splats because they lack the necessary feature points for the software to anchor against. While adding markers or taking a large number of photos could help, you should be prepared to perform some post-processing edits to mitigate the "cloudy wall" effect.

4: Capturing a scene: the value and limitations of video

When capturing a space for a 3D model, taking individual photos with a high-resolution camera is often the gold standard. But what about video? Can you create a good model from video footage alone? We tested this by using 4K video from a camera on a four-meter selfie stick to capture the entire exterior of the Allee Willis house.

The entire exterior of the Allee Willis house, including the roof, was reconstructed as a Gaussian splat using individually extracted frames from video, thanks to the help of a selfie stick! (Flat satellite imagery was used for the surrounding homes for context.)

The two critical factors: light and resolution
Our experiment showed that video can be a viable option, but success hinges on two key factors:

• Light: The biggest concern with using video is motion blur. Since video is just a rapid series of images, any camera shake or fast movement can blur individual frames. This blur makes it very difficult for the software to find the unique feature points it needs for a successful reconstruction. In our test, there was plenty of natural light, which allowed for a faster shutter speed, effectively eliminating motion blur.

• Resolution: Even with great lighting, video has inherent resolution limits. While 4K video resolution (3840x2160 pixels) offers a good level of detail, it typically can't compete with the resolution of a modern full-frame DSLR or mirrorless camera. As a result, a Gaussian splat created from video may not be as sharp or detailed as photos using a full-frame sensor. If video-based Gaussian splatting is an approach you’d like to pursue, we’ve seen examples where a rig of multiple video cameras could produce better results than one video camera alone.

Key takeaway
Despite the resolution limitations of 4K video, we were pleasantly surprised by the results of our test. When shot in a well-lit environment that minimizes motion blur, a 4K video can be an effective way to capture a scene. While the final Gaussian splat from a 4K video may not have the same level of detail as one created from high-resolution still photos using a full-frame sensor, it can still provide a good representation of a space.

5: Prioritizing your shots: the art of triage in complex scenes

The biggest challenge with Gaussian splatting is that any angles you miss while photographing an object will likely result in issues in your final 3D Gaussian splat. Photographing a single object is easy. But what happens in a complex environment?

The all-encompassing challenge

It's relatively easy to capture all the necessary angles for a single, isolated object using a comprehensive approach like the classic circles/dome trajectory or the concave half-shell trajectory. However, this becomes much more difficult when you're trying to photograph an entire room filled with hundreds of objects, like a museum exhibit or a crowded open office. In these complex environments, it's virtually impossible to get perfect coverage of every single object from every angle. This is when you must be strategic with your approach.

Strategic triage
To handle this, you need to triage your photo capture. This means prioritizing your efforts to ensure you get the best possible coverage of the most important objects in the scene. Take a full range of photos of the primary subjects of interest. For secondary or background objects, it may be necessary to take fewer photos or use less comprehensive trajectories. By making these choices, you accept that some parts of the scene may not be perfectly reconstructed, but you guarantee that the most critical elements will be captured with sufficient detail and accuracy.

Key takeaway
The angles you don't photograph may come back to haunt you, as there will be missing information in those areas. However, in complex scenes with many objects, it may be difficult to capture every angle. Therefore, it's essential to strategically prioritize and triage your photo capture to ensure the most important objects are well-represented, even if it means sacrificing some detail on less critical elements.

6. Low light? No problem! Shooting at higher ISO

When working with Gaussian splats, using a higher ISO (a measure of light sensitivity as determined by the International Organization for Standardization) setting can be a viable option in low-light conditions, as there is still often enough information for the structure from motion (SfM) process to succeed. While higher ISOs introduce more noise, a careful approach can still yield successful 3D reconstructions. Of course, we don’t recommend resorting to a very high ISO until you’ve first made reasonable adjustments to shutter speed and aperture. 

Structure from motion at higher ISOs?
One of the key techniques behind 3D reconstruction is structure from motion (SfM). This process works by identifying and tracking unique feature points in your photos. These could be anything from a unique mark on a floor to the corner of a window frame. The software identifies key points in each photo, matches them between different photos, and then uses those matches to mathematically reconstruct the 3D scene. This process is crucial for creating the point cloud data that Gaussian splats use to render a scene.

Higher ISOs, which increase a camera's sensitivity to light, introduce more digital noise, which may affect the accuracy of the SfM calculations. However, based on our experiments, this noise does not necessarily prevent the SfM algorithm from finding enough information to create a workable 3D reconstruction. The key is to take the photos carefully, minimizing motion blur and other potential issues.

Additionally, you may wish to consider using a tripod to afford slower shutter speeds when there is extremely low light. This will increase the amount of time for photography substantially, but the higher quality results could be worth the extra effort.

Key takeaway
While lower ISOs are always preferred for a cleaner capture, structure from motion can still successfully generate a 3D reconstruction from carefully taken, high-ISO photographs, making it a viable option for creating Gaussian splats in low-light environments.

7. Speed vs. quality: the case for JPEG

When capturing a scene, the quality of the photos is paramount, but sometimes you don't have the luxury of time.

In high-pressure situations, such as a busy soundstage with tight deadlines, a team may have a very limited window to capture multiple setups. In these cases, it can be beneficial to sacrifice capturing photos in RAW format and instead shoot in a high-resolution JPEG. While RAW files offer greater flexibility for post-production editing, because of the larger file size due to increased dynamic range and image quality, they take longer to write to the SD card, which can slow down a shoot considerably. By opting for JPEG, you can significantly increase the speed at which you take photos.

The main risk of this approach is the lack of flexibility in post-production. Unlike a RAW file, a JPEG applies in-camera processing such as exposure, white balance, and saturation and then compresses the image. This makes it harder to correct mistakes later. Therefore, if you choose to shoot in JPEG, it's crucial to ensure your exposure is set correctly. This will help you achieve the best possible result despite the time constraints.

Key takeaway

In time-critical situations, shooting high-resolution JPEG photos can save you valuable time over RAW, allowing for a faster photo shoot. The trade-off is that you must ensure your camera's exposure is perfect, as you will have very little latitude to adjust the image later.

8. The 5000 Photo Challenge: when more ������’t better 

When it comes to creating a Gaussian splat, there's a practical limit to the number of photos you can use. As the photo count increases, so does the time required for the structure from motion (SfM) calculations. Beyond 5000 photos, there is a real risk that your computer may run out of memory, even if you have the time to wait for the structure from motion computations.

The 5000 Photo Threshold

Based on our practical experience, SfM computations become significantly slower after about 5000 photos. For this reason, it's often more efficient to use a targeted number of photos for your reconstruction, depending on the visual fidelity requirements.

Here are some general guidelines for photo count based on desired visual fidelity:

• Very high fidelity: 3000 to 5000 photos

• High fidelity: 1000 to 2000 photos

• Medium fidelity: 500 to 900 photos

• Lower fidelity: 100 to 400 photos

• Very low fidelity: 20 to 50 photos

The training process

The total time it takes to train a Gaussian splat is a direct function of the number of photos. More photos mean more data for the algorithm to process, which increases the training time. The resolution of the photos is also a factor, and downscaling the photos will help speed up the camera pose calculations. Smaller reconstructions may take a few hours, and complex scenes can take many hours, or even several days. A key factor to increasing the probability of success is ensuring a very high degree of overlap between your photos. Without sufficient overlap, the SfM algorithm won't be able to accurately align the images, leading to a failed reconstruction.

Key takeaway

For efficient and reliable Gaussian splat reconstructions, it's best to work with a targeted photo count rather than using as many as possible. Exceeding 5000 photos significantly increases processing time for the structure from motion calculations.

9. Ask permission!

While capturing photos for a Gaussian splat, your unusual movement pattern can draw attention. To avoid any potential misunderstandings or legal issues regarding copyright, ask for permission from the property owner, manager, or security personnel before you start. When you capture photos of a location or a specific object, you are also capturing the intellectual property of its owner. This could include copyrighted artwork, product designs, or even the unique architectural layout of a building. Without explicit permission, you could be infringing on their rights.

A simple and clear explanation of your project can often be enough to get the go-ahead, saving you from a potential misunderstanding down the line.

Key takeaway

Always try to get permission before shooting. Capturing photos for a Gaussian splat requires a distinctive movement pattern that can draw attention, so being prepared to explain your process can save you from interruptions and potential issues.

10. A note on editing 

Editing Gaussian splats can be easy or difficult, depending on what you need to accomplish. Simple edits are quick and efficient, but precise and complex workflows remain challenging. 

SuperSplat 
is an indispensable tool for viewing and editing splats. It makes basic workflows easy, allowing you to select, edit, delete, move, rotate, and scale splats. For example, you can select a spherical region of interest and quickly delete everything outside of it, or combine separate splats into one file. SuperSplat can even edit the color of splats and works on both a desktop and an iPad.

While SuperSplat is excellent for simple edits, more complex workflows requiring precision are challenging. For a large project like the Allee Willis house, which involved many tens of thousands of photos, it was impossible to process everything at once. We had to break the home into sections, but we found that putting them all back together to form a cohesive, seamless Gaussian splat was a major hurdle. This is because precision workflows with Gaussian splats are currently not as straightforward as similar operations on polygonal meshes in software like Maya or Blender.

The challenge of precision

While basic edits are straightforward, achieving high-precision edits is much more difficult. This is because Gaussian splats are inherently more complex to edit than polygonal meshes. Splat "brush strokes" are interwoven, meaning one change can lead to a cascade of problems. Fixing a splat can feel like mending a frayed straw weave hat – pulling on one long splat to fix a visual artifact can cause a chain reaction that disassembles other parts of the scene. However, the field is advancing rapidly, and new research is being published frequently. In the future, editing Gaussian splats will undoubtedly become much easier and more intuitive.

Additionally, current Gaussian splat editing software struggles to sort the depth of separate splat objects in the same scene. This means that a Gaussian splat object that is partly or mostly behind another splat object may be rendered partly or mostly in front of that other object. Consequently, it becomes more challenging to understand the placement of Gaussian splats relative to each other in space.

This difficulty becomes most apparent in high-precision workflows, such as:

• Aligning splats that are co-located precisely against each other.

• Merging splats with critical boundaries seamlessly.

• Fixing visual artifacts like "cloudy walls."

Key takeaway

Simple edits are easy and efficient, but precise and complex workflows remain challenging due to the interwoven nature of the splats. While current Gaussian splat editing tools are easy to use for basic tasks, precision editing requires significant time and manual effort.

Bonus tip 1: Gaussian splat software recommendations

The following is a partial list of software/services that can be used to process photos or video to create (train) Gaussian splats:

• is a solid NeRF and Gaussian splat solver, but it will take time to compile and set up. A secure option, as it runs locally on your computer.

• provides visual feedback while the Gaussian splat is being trained, and it runs locally on your computer.

• Gaussian splat cloud solver will take up to 1000 photos (up to 2000 for photogrammetry).

• RealityScan doesn’t train Gaussian splats, but it is very reliable for structure from motion calculations for cameras for Gaussian splats, as well as for photogrammetry.

As always, be sure to read the terms and conditions to make sure it’s a good fit for the needs of your project or organization. 

Bonus tip 2: turning a Gaussian splat into a digital twin with OKO

OKO is our game-changing platform that can use Gaussian splats as a foundation for the creation of a digital twin. It allows teams to enter spaces and collaborate in real-time, on desktop, mobile, and XR web browsers, iOS, and even Unreal Engine. We’ve used OKO for collaboratively planning complex projects in real-time, and Gaussian splats are integral to our workflow.

OKO workflow for digital twins using Gaussian splats

Phase 1: capturing the space
The journey begins by capturing a physical space using a camera. These photos are processed to create a Gaussian splat. We also recommend creating a photogrammetry mesh from the photos for the next phase.

Phase 2: building the digital twin
The Gaussian splat is used as a foundation to create a 3D digital twin. Photogrammetry can be used as a collision mesh that is aligned with the Gaussian splat, as splats do not inherently support collision. Photogrammetry as a collision mesh works remarkably well for stairs, inclines, walls, bridges, and passageways.

Phase 3: bringing the space to life
Once the digital twin is built, teams can add other objects and interactions to bring it to life.

Key takeaway

OKO turns a scan of a real-world location into a shared, collaborative digital space where people can meet and work together from anywhere. It's a powerful way to democratize digital twin creation, and there’s nothing quite like experiencing a new space with friends, colleagues, clients, and collaborators.

Final thoughts

Gaussian splats represent a major leap forward, offering stunning aesthetic quality that traditional photogrammetry and LiDAR simply can't match. While we are more familiar with working polygons, in our experience, the challenges of editing Gaussian splats are worth the effort.

The visual benefits are compelling enough that we’re not waiting for the "next big thing." We're all in on Gaussian splats. With a rapidly growing ecosystem and a passionate research community, we're confident that improved tools and techniques will soon make the editing process much more efficient. And we're excited to be part of the community, pushing the boundaries of what's possible.

Credits

Creating these Gaussian splats would not have been possible without Vince Beggs and Prudence Fenton from the Willis Wonderland Foundation inviting us to create a Gaussian splat digital twin of the Allee Willis house.

Rusty Blazenhoff, Sean Welch, and Ari Bird, also from the Willis Wonderland Foundation, helped us in numerous ways during the photo shoot, shared countless stories about Allee Willis, and taught us so much about the history of Allee’s house.

While Fabian Flores’ Gaussian splats are not shown here, Fabian did an amazing job of Gaussian splatting other parts of the Allee Willis house. Fabian also worked with Julio Salcedo to digitally scan and measure the entire house for scale reference.

Julio Salcedo, Troy Chu, Hanuel Kim, Teruhisa Yoshida, and Darren Bunch did an excellent job of lighting the house for the interior shots.

Shriya Wani helped to add scripted interactions in OKO.

Alessio Regalbuto, Sam Birley, and Adrian Meredith did a superb job of enabling OKO iOS, Unreal, and Web to render Gaussian splats for collaborative visualisation.

Ashley Bravo, Johanna Gardner, and Keith Michaelis are wonderful producers who helped make sure everything ran smoothly.

And last, but not least, Lap Luu, Kevin Mullican, Daisy Leak, Alicia Mirelez, and Daksh Sahni supported this Gaussian splat project from leadership roles at �㽶��Ƶ.

Addison is an innovative audiovisual designer and creative director, shaping the future of entertainment through XR music experiences and cutting-edge visual storytelling. Drawing on a background in dance and choreography, he creates reactive lighting, kinetic environments, and interactive motion effects. Addison has designed LED wall visuals for major artists including Lil Baby, Ice Spice, Snoop Dogg, and Nickelback, and led concert experiences for Karol G and Sebastián Yatra. His work spans virtual production and lighting design for projects such as Amazon’s Fallout and HBO’s Westworld. He most recently directed the new music video for Daft Punk’s “Contact”. 

Tell us about your role at �㽶��Ƶ

I provide creative direction for audiovisual projects – ranging from traditional concerts to installations to interactive game experiences. I work with musicians to figure out how we can build worlds around their sound, mythos, fashion, influences, etc. I spend a lot of time tracing what makes an artist meaningful to their audience – why does the audience love this person or band so much – and using that to imbue these worlds with a heart and a soul. 

What’s the most recent project you’ve worked on?

The last project I worked on was Daft Punk’s official music video for “Contact”! This was a total love letter to the band that got me into electronic music, and I’ve never been so excited to wake up every day and work on something. So many people who worked on this project hold a deep reverence for this band and this song, and I hope that passion is visible in the final video. 

What made you decide to pursue a career in this field?

During the pandemic, I was working as a Pipeline Engineer – not doing much art or design at all – but in the background, I was discovering this really incredible music scene in . This is a totally grassroots community of people who teach themselves Unity, design virtual stages, and then throw raves for their friends. People show up with full motion capture suits to dance, people live-code music or visuals and inject them into the world – it’s truly cyberpunk and unapologetically queer (you can be whoever you want as a digital avatar). 

I was very inspired by this scene and similarly started teaching myself shaders, lighting design, DMX, and stage design. After a lot of bothering coworkers with “hey, look at this virtual rave thing”, I transitioned out of engineering into design at �㽶��Ƶ. 

There’s moments at shows – virtual or physical, doesn’t matter – where everything that the performer, the stage designer, the lighting team, and the VJ are doing is hitting in perfect sync at the right time – and you get this instant where you appreciate how incredible it is to be alive in the moment, surrounded by these people, and then you just want to dance. I’ve been lucky enough to experience that feeling quite a few times in my life, and now all I want to do is pass that on and create those moments for other people. 

What’s the biggest lesson you’ve learnt in your career?

I learned early on from a mentor that you have to constantly fight and advocate for yourself as an artist. I never shut up about how much I care about music and audiovisual art, and I don’t compromise on the principles that I care about. 

I’m lucky enough now to have a lot of people at �㽶��Ƶ on my side helping secure more music projects, but starting out in a competitive creative field, without my foot in the door, nobody was dropping opportunities in my lap. I’ve been pretty aggressive about making art that is meaningful to me, and then contacting people that I think it would resonate with, or putting meetings on people’s calendars explaining how my work could be useful to them. 

On my first day as a pipeline engineering intern at �㽶��Ƶ, with zero studio art experience, I sent our CEO a music video I had made in my free time. It definitely wasn’t any good… but I think that unrelenting, brazen sensibility of ‘let me make cool stuff with you’ has helped me get to a point where I can do art/design full-time now. 

What, or who, inspires you?

I spend so much time studying and making visual media that I’ve inadvertently become more drawn to and affected by narratives and text-based media. I don’t cry that much when I’m watching movies, but I cry all the time when I’m reading. 

I think the mechanical process of reading and staring at text, since it’s so minimal from a sensory standpoint – just black words on paper – can be even more immersive than visual media. You’re letting your brain fill in all the blanks rather than being told exactly what something looks like, and the way your brain fills in those blanks is often more personal than if you were passively watching something. 

My favorite books are the ones with unconventional structures that utilize text as a medium and do things that wouldn’t be possible in film or television. Stuff like or ‘new weird’ authors like , where prose is shifting back and forth with poetry, and there’s this beautiful and incomprehensible world unfolding in the back of your mind, but really you’re just staring at a page. I love and seek out that feeling, and a lot of what I design is inspired by that kind of dream-like and amorphous imagery. 

What’s your favourite thing to do when you’re not working?

Previously, my main creative outlet was dancing in and choreographing for megacrews in the SoCal collegiate hip hop scene. This community was highly intense and competitive. We would spend a lot of late nights ‘cleaning’, ensuring everyone’s arms were hitting the exact same angle at the same time. You’d have to be sharp and precise, and unified. 

I loved these experiences and it was a great foundation, but in the last few years, as a side effect of attending a lot (...hundreds…) of warehouse techno shows, my relationship with dance and choreography has changed pretty significantly. Nowadays, I love to just hang out in the back of underground events with other dancers and just freestyle and flow for hours, completely unstructured. It’s very meditative – when you close your eyes and sink into the pocket of a groove and the stress leaves your mind and you’re just letting the music move you. 

If you were on a game show, what would be your specialist subject?

Highly specific categorization of niche electronic music into subgenres. In my heart of hearts, I know that genres are arbitrary and musicians should create whatever they want to without thinking of boundaries or rule sets. Still, I find it incredibly satisfying to sort and identify music. And sometimes you learn that all of your favorite tracks fit into something called ‘hardgroove’ and then it’s suddenly a lot easier to find similar stuff.    

What are you reading/listening to/watching right now?

I’m reading , which is a collection of Jorge Luis Borges’ short stories and fables – most famously containing . Another amazing example of utilizing text as a medium and conveying ideas that couldn’t be adapted to film. 

I’ve been listening to a lot of , he’s an incredible industrial techno producer from the UK with very contorted, broken-sounding production. Sort of like if you take a classic breakbeat and then throw it into a meat grinder and then feed the result to a Lovecraftian monster, what would that sound like? (It would sound like Blawan.)

Tell us something that would surprise us!

When I retire, I want to be a park ranger.

When I was tasked with designing a high-energy, chaotic race track for the Metallica event in Fortnite, I knew coming up with memorable ideas was just one piece of the puzzle. The real challenge was turning those ideas into something that worked, both technically and visually. 

In this article, I’ll share how we balanced creativity with execution – guiding players, controlling pace, and creating moments that felt unforgettable. Whether you're designing a level for a game, a virtual environment for an event, or even a user interface for an app, the goal is the same: to bring bold ideas into reality through smart, intentional design.

Design brief

The initial brief was to build something chaotic, collaborative and immersive. While that sounds straightforward, bringing those ideas to life within a race track was a new experience for me. 

To begin, I needed to understand not only what makes a track functional, but also memorable. This led me to the research phase. I jumped headfirst into analyzing a variety of tracks, from F1 circuits to go-kart tracks and, of course, video game levels! Part of this was to understand what looked good, but I also wanted to understand what felt good.

One of my key takeaways during this stage was the importance of escalation – the track needed to gradually increase in intensity and complexity to keep players engaged and motivated. Achieving that ultimately came down to the layout of the track.

Track layout design 

Although there were multiple avenues to explore during the initial design phase, the three elements that stood out the most to me during the research phase were: 

• Sharp/banked turns

• Track splits

• Rhythm and pacing

All of which had previously been used to build tension and create a strong sense of progression over time. For example, both and from Ubisoft use sharp, banked and even looped turns to keep the track exciting and engaging for players.

Sharp and banked turns

This feature demands more focus from the player. No matter how fast or slow they’re driving, they’ll have to consider braking, drifting or carefully positioning their vehicle to achieve a smooth turn. This helps naturally immerse players, making them feel more involved in the moment-to-moment gameplay.

Straight tracks help build momentum, but without turns, races can feel boring and lack challenge. Turns break up the speed and add variety. By mixing their shape, angle and placement, we were able to create a rhythm that kept players thinking and reacting. This balance makes races more fun and gives players room to improve with practice. The GIF below demonstrates how our turns add excitement and maintain player engagement. 

These learnings on sharp and banked turns were a helpful starting point in building the track, but actually implementing them in engine was a different story!

Attempting to create smooth, playable turns was where the real challenge started to appear. With a banked turn, you can loop it round – really take it as far as you want. However, doing this comes with its drawbacks. In my experience, any turn over 90 degrees led to collision issues, where vehicles didn’t behave as intended – bumping and even causing vehicles to flip. This was purely down to the fact that the spline mesh (a flexible mesh that follows a curved path you define using control points) would often twist or stretch unnaturally, which led to gaps and overlaps in the track.

As a workaround, I found it was beneficial to break sharp turns into smaller segments with evenly spaced control points and smooth tangents. This ultimately ensured I had greater consistency and control over the track layout. 

Key takeaway: Turns demand more focus from players by adding challenge and variety, but they also require careful design and implementation to avoid technical issues like collision and mesh distortion. 

Track splits

Track splits can be designed to create meaningful choices that impact the flow of each race. Instead of locking players into a single path, splits give them the opportunity to decide – left or right? This decision encourages players to think about the consequences mid-race, adding tension and excitement.

From a gameplay perspective, track splits aid in reducing crowding, encouraging high-speed decision making and introducing meaningful choices that impact the flow of every race. Each route offers something different – like a sharper turn, a shortcut, or a safer path – giving players more ways to react in the moment. These choices allow players to lean into their own playstyle, making each lap feel personal. They also encourage replayability, as players return to try alternate routes and improve their times. See below how one path feels more intense and demanding, while the other offers a steadier pace. 

Like with sharp/banked turns, I often noticed gaps and overlaps in the mesh wherever I placed a split, especially where paths would diverge or merge. Not only did this have an impact on the visuals, but the gameplay too – vehicles would bump and bounce along the way. 

The solution came down to careful placement of the control points on each spline. Ensuring each control point was evenly spaced and tangents were properly aligned was crucial to preventing those gaps and ensuring a smooth transition between paths. 

Key takeaway: Track splits encourage quick decision making, giving players more control and variety. However, they require thorough testing to prevent mesh gaps and gameplay disruptions. 

Rhythm and pacing 

One of the core goals for this project was to include beat-synching elements. We achieved this by mirroring the emotional highs and lows of the music through player movement. Every ramp, drop, and curve was intended to flow with the music, making the player feel like they were a part of it. 

For instance, as the music reached its climactic point towards the end, we decided to design a ramp that launched players into the air at the same moment. Imagine that feeling of flying through the air as the music hits its peak! Adding these events creates a sense of payoff and achievement – really transforming a musical beat into a significant gameplay moment. 

With that said, synching these kinds of moments to player actions was not easy. One design choice that was made early on was to add ‘steps’ to the track layout that the player would hit in time with the music beat. But this led to each playtest feeling inconsistent and unpredictable, which is the opposite of what we wanted. 

This is where balancing creative vision with technical requirements really came into play. As a functional alternative, we moved towards widening the scope – using broader rhythmic cues and environmental beat-synched elements, rather than restricting the player to specific timings. This led to elements such as: lava geysers erupting on the beat, lights on the track pulsing rhythmically, and even adjustments to the layout that encouraged bursts of speed. 

Key takeaway: Aligning player movement to music enhances immersion, but relying on wider rhythmic elements over strict timing ensures consistency and flow. 

What made the final track memorable? 

When the visual, musical, and gameplay elements finally came together, it became clear we had created something special – an experience that felt immersive, emotional and unforgettable. 

Visual identity

There was a cohesive visual identity throughout the track: a dark, grungy aesthetic with fire, lava and glowing lights that synced to the beat of the music. We needed it to feel powerful, but technically, it had to run smoothly. So we were careful with our placements. Lava geysers and jets were used selectively, and instead of overloading the track with constant effects, we used them strategically to highlight key areas to guide the players’ attention and sync to the beat. This approach ensured players felt fully immersed in the experience, without compromising performance. 

Engaging player moments

Several chaotic elements made it into the final iteration. Jump sequences, swerving through lava, chasing down the band, and even syncing certain moments to the beat. However, too much engagement would have been overwhelming and distracting for the player. So we carefully spaced out key moments to ensure each had a real impact. We wanted the experience to be chaotic, without causing confusion. So with careful balance, we were able to create a track that felt energetic and captivating, without overwhelming players. 

Learnings

One of the biggest takeaways from this project was understanding how to align a creative vision with technical demands. Ambitious ideas like beat-synch elements, high-energy racing moments, and visually striking aesthetics all succeeded because they were supported by intentional design choices around structure, pacing, and execution. 

The process has shown me that an impactful experience doesn’t necessarily come from doing more, but from making purposeful decisions that support both the creative goals and technical constraints. 

Dan is a Creative Director at �㽶��Ƶ with over 25 years of experience in the games industry. He’s worked on a wide range of AAA games, including Hitman, Deus Ex, Tomb Raider and more! In 2015, he founded Thunderbox Entertainment – pioneering AR and VR technology for tabletop gaming – before moving on to direct the critically acclaimed,  telekinetic shooter for PlayStation VR2. At �㽶��Ƶ, Dan has brought his years of experience, immense talent, and fabulous sense of humor into every project he gets his hands on.

Tell us more about what you do at �㽶��Ƶ.

Sure! I’m a Creative Director… which sounds pretty fancy, but basically my job is to help the team unleash their full creative potential on whatever wonderful project they might be working on. This usually consists of listening to their collective ideas and distilling them into a shared vision, before providing executional focus to deliver that vision.

You’ve got quite an impressive CV! What made you decide to pursue a career in this field?

Ever since I was young, I’ve always loved two things: entertaining people and technology. Video games, and other forms of immersive entertainment, are the fusion of both those things, so I kind of just did what I loved and took it from there. I’ve been lucky enough to work on some amazing games… but also some downright awful ones. Fortunately, nobody remembers the bad ones, and they were all phenomenal learning experiences.

A little birdie told us you founded your own company. Tell us a bit about that.

I did! That was a lot of fun. I really love board games, and thought there was a cool opportunity to share that passion with a wider audience by bringing the best of them to mobile platforms, so I started Thunderbox to do just that. We made a few great board game apps before shifting focus to AR, which was super new at the time, and then VR, which landed us a coveted for our VR adaptation of . As far as I know, we’re the only game dev studio to ever win one. Running your own company is amazing from a creative standpoint, and it forces you to step outside of your comfort zone every day… but it’s certainly not for the faint of heart!

What’s your favourite thing to do when you’re not working? 

Playing with my crazy dog, Matilda. She’s a rescue from Texas. I call her the Treat Mafia because she’s very good at shaking down other dog owners for biscuits. She’s a Blueheel / Coonhound / Greyhound cross – it’s like somebody tried to genetically engineer the perfect squirrel-seeking missile.

Aside from that, all the musical projects we’ve done recently at �㽶��Ƶ have inspired me to get back into music production, so I spend a lot of time in my studio at home creating melodic Drum & Bass under the moniker . 

What values or principles do you consider non-negotiable in both your personal and professional life?

Honesty, compassion, and respect. I try hard to give those every day (not always successfully), and truly appreciate any reciprocation. A sense of humour doesn’t hurt either.

How do you approach challenges and setbacks?

Pretty much everything is a challenge on some level or another. If it ������’t, then you’re probably doing something wrong. Very often, adversity can be the mother of invention, so it’s good to be challenged.

Setbacks are frustrating, but they happen to everyone, so I think it’s important to be objective, identify what went wrong, and adjust your approach; that way they become positive learning experiences rather than annoyances.

If you didn’t have to sleep, what would you do with the extra time?

Well, everyone else would be asleep… and it would be dark… so I’d probably wander around looking at the stars, whistling quietly to myself. And maybe go for a swim in the sea if it was warm. That or become Batman.

If you could have any other job in the world, what would it be?

Waterslide tester.

What’s something you think everyone should do at least once in their lives?

I lived in Canada for a decade or so, and I highly recommend moving to a foreign country for at least a year. It gives you a totally different perspective on where you come from on a global scale, as well as a deep appreciation for cultural diversity.

Or get a dog. Dogs are the best.

What has been your proudest/standout moment while working at �㽶��Ƶ?

Every day is a standout moment. I’m constantly amazed by the work the teams here at �㽶��Ƶ do. Every single day, somebody shows an amazing piece of work: from super-tight level design to a small audio tweak that totally transforms the vibe of an experience, and everything in between. It’s an honour and a delight to work with such a talented bunch of folks.

Have you ever looked at a shot from a game or movie and thought, “Wow! That looks amazing, but I’m not sure why”?  Well, that’s the power of good composition. 

But what is composition? And what is it that makes one particularly good? 

A great scene relies on several key elements: lighting, color, a compelling subject (maybe even a puppy), and crucially, composition. 

As an artist – whether you work in traditional painting, 2D, or 3D (for gaming, interactive, or film – you need to understand what makes an image work. We instinctively react to images, sensing whether they feel happy, scary, or calm. It’s your job to use those cues intentionally.

Composition is the thoughtful selection and arrangement of all the elements in your scene. Understanding this skill is an essential tool for any creative.

In this article, we’ll discuss the importance of composition and the tools you can use to help you create amazing, memorable images!

What do we want to achieve with composition?

Generally, there are three main things composition is trying to achieve:

Firstly, we can use composition as a tool to control readability. We can guide the viewer’s eye so they focus on the intended elements or actions. This helps with understanding the image and reduces visual clutter. Even the most compelling subject can be lost in an image that’s not properly composed.

Secondly, the composition can set the mood. Colors, lines, shapes, and textures can all contribute to the overall emotional tone.

 And lastly, the composition should be interesting. There are countless ways to make a scene more appealing or, if the story calls for it, deliberately ugly, cluttered, or chaotic. Creativity often comes from seizing an opportunity to try something original. Rules can be broken, and the same story can be told in many different ways. Having a strong vision or a unique expression can lead to wonderful, unique work.

Understanding the basic rules of appealing composition is a great starting point. Once we know how to structure an image, we can start breaking those rules to create something truly compelling.

Directing attention

In general, our eyes are drawn to high-value contrast. In this image from the game Ori and the Blind Forest, there’s a lot of visual clutter going on, but our eye is immediately drawn to the characters in the center. This is done by creating a big difference in value between the character in the foreground and the mountain in the background. 

You can check the values of an image by turning it black and white. In this example, the values of the characters at the front range from almost black to almost full white. This contrast draws our eye to the intended focal point of the image. It communicates to the player that this is the most important thing in the scene. If you’re looking to recreate the value levels in this image, a great way to reduce the contrast of the background is to add fog.

However, something else to consider is that contrast ������’t just value; it’s also impacted by color choices. In the first image, we can see that the background is orange and the foreground blue. These colors are complementary – they contrast each other strongly.

Another good example of this can be seen in the game Hades.

The first thing that stands out in the above image is the orange monsters and our red main character. They achieve a heavy contrast against the dark, muted green background due to their distinct colors. It's important to note that the main elements are very bright and saturated, while the background is deliberately subdued. Since the most saturated part of an image draws focus, the bright green pillars are intended to guide the viewer's eye.

However, the monsters are also bright, saturated, and a complementary color. On top of that, they move, so they'll draw focus even more than the pillars. The player's first read of this area will therefore be the monsters, the second the pillars, and thirdly, everything else. Paying attention to these visual hierarchies makes for a better player experience, as it gives you control over their focus.

Another simple way to direct focus is brightness. If you want the viewer to look at what’s on the table in a scene, put a light on it! This immediately feels like something important that the viewer (or player) will want to investigate.

Sometimes a subject can stand out just by its shape, particularly when it feels like it doesn’t belong in the landscape. Take this machine, for example; it stands out as a threatening presence in a peaceful environment.

Lines are another key composition tool, especially those that point at your subject. We can see in this example from Kena: Bridge of Spirits that there are many lines in this environment. The branches pointing at the center are all functioning as arrows pointing at our figure. It’s making him look bigger and more menacing.

An object with a sharp point, like the sword in this scene, is a strong tool. It functions similarly to a line by directing the viewer's gaze, but the sharp tip introduces an element of tension and focus that demands attention.

Eyes, faces, and even hands have a similar effect. As human beings, we are very attuned to reading facial expressions and body language. If there’s something even remotely human in an image, our eyes will be drawn to it, follow its line of sight, and interpret the hand gestures.

An interrupted pattern is a powerful artistic tool. When a sequence is broken by an odd element, it immediately draws our attention. A regular pattern can function almost like a solid surface or background in a scene. Therefore, anything that disrupts this pattern will naturally become the focal point.

Lastly, movement. We all know that movement is relative, so if your subject is moving with a background which is standing still, it will stand out. Alternatively, if the background is moving and your subject is the only thing that’s not, that works too. We can use all of these things to guide our focus to subjects in our work!

Examples of compositions

Now that we know how to direct focus, it’s time to start playing with composition to tell our stories and set a mood. There are many tools we can use to achieve this, let's take a look! 

Points

One of the simplest compositions involves reducing the detail in a scene to only one point of interest. The eye is naturally pulled towards this focal element because it stands out as the only visual anchor.

Lines

As mentioned earlier, lines can guide a viewer's eyes to a subject.  Beat Saber, a VR rhythm game, uses concentric lines to keep your attention straight on the center of the action. In VR, you have the option to look in any direction, but because of the way it’s framed, you’re hypnotized. 

Different sorts of lines will also affect the mood of an image. For example, in this image from Planet of Lana, we can see how horizontal lines can evoke a feeling of calm and serenity. 

Whereas in this image, there is a stark contrast between the serene horizontal line of the landscape and the diagonal way the aliens are invading. Clearly, they don’t belong there and disturb the peace.

A more vertical line will look more imposing; it makes the subject look small, like the task ahead is daunting. In the case of this image from The Witcher 4, they’ve played with scale, making the character feel very small compared to the vast cave.

Meanwhile, curved lines can look either elegant or chaotic depending on whether they bend in an organized or disorganized way. The curved lines in the first image create a stressful atmosphere, whereas in contrast, the second image has a calm, clean aesthetic.

Camera angles

Depending on the position of your camera, you can communicate different dynamics. Putting the camera at eye height feels pretty neutral, as though the viewer is on the same wavelength as the character in the scene.

However, when we put the camera lower than the character, suddenly they look imposing and scary; an intimidating presence in the scene.

A high camera angle achieves the opposite effect: it diminishes the subject. This perspective makes the character appear less powerful or physically smaller, suggesting they are humbled, not in control, or in immediate danger. This angle can also create the feeling that the subject is being watched.

Space in the frame

The space where the character is looking is known as the “lead room” or the “looking room”. Typically, you want to keep the space in front of a character larger than the space behind. This visually prevents the character from feeling confined, and is the standard approach for simple scenes or conversations because the resulting composition feels natural. It just feels nice.

Doing the opposite feels off, like the character ������’t really there mentally. It creates tension. This imbalance can be used intentionally to suggest a feeling of unease or mystery.

The space taken up by the subject is called positive space; the rest is negative space. Playing with the amount of space around the subject can give different effects. We feel relatively close to characters that are big in the frame.

In contrast, a larger amount of negative space makes our character feel small, and the world more dominant. This can often help a scene to feel more serene.

Making your character smaller in the frame can also help create an establishing shot. It shows the viewer where they are and what the setting is.

We can also fill the frame by getting really close to our character. It can feel intimate, uncomfortable even, placing us in direct connection with the emotion that the character is experiencing.

The space above a character’s head is called “headspace”. Typically, it’s kept to a minimum, but in Mr Robot, it was used to emphasize the character’s isolation. This creates a feeling of imbalance. Just like dissonance in music makes us crave the resolution of consonance, this kind of framing builds tension and drives the story forward.

In gameplay, your camera options are different. The camera positions are mostly determined by gameplay considerations that usually outweigh the artistic sensibilities, but these techniques can still be used, especially in cutscenes or narrative-driven moments.

Depth

Depth of field is a powerful tool when it comes to composition. In film or photography, depth of field can be adjusted by changing the aperture on your camera. Aperture determines how much of the scene stays in focus. In games or other virtual spaces, the same tools are generally available within the renderer or engine you use. 

Adjusting the depth of field can give different effects. Having a narrow depth of field can make the viewer focus on a specific object. For example, in the image below, the depth of field enhances the storytelling by making the viewer focus on the character's phone rather than her face.

On the other hand, a large depth of field can make a scene feel cluttered. In this shot, we see a woman blending into the pet shop, which works narratively, but it can be visually overwhelming.

Another reason to use a large depth of field can be to show the scale of a city or an army. Allowing the viewer to see the large scale of something can create a sense of awe or intimidation, depending on the narrative need! 

Something to note is that establishing shots often have a large depth of field, while the shots with the actual action have a narrower depth of field.

Another fundamental way to enhance visual depth is to establish clear foreground, midground, and background layers within a shot. Giving viewers these distinct layers to explore makes the image feel more dimensional. Generally, a shot with this layered structure will be more visually engaging than one presenting a single, flat layer.

However, sometimes a director will intentionally create an entirely flat set, which gives a very interesting, distinct feel. This demonstrates that there's more than one way to make an image compelling. What works beautifully for one project may not work for another, and seeing that wide variety of creative possibilities is what makes the whole process so engaging.

Overlaying grids

The Fibonacci spiral

Mathematically, the Fibonacci spiral is constructed using squares with side lengths corresponding to the numbers in the (0, 1, 1, 2, 3, 5, 8, ...), where each number is the sum of the two preceding ones. A quarter-circle arc is drawn within each successive square, connecting its corners to form a continuous spiral that expands outwards. The Fibonacci sequence approaches the golden ratio as the numbers increase.

In nature, this is the most efficient way to arrange seeds or petals, which is why we see spirals in things like sunflower heads. Once you start looking, you’ll spot the spiral everywhere!

It turns out our eyes like to follow this direction. It feels pleasing – dynamic, yet orderly.  For a very long time, the Fibonacci spiral and the golden ratio have been considered aesthetically pleasing and harmonious. 

Leonardo Da Vinci, for example, used them heavily. He even wrote a book, , discussing the golden ratio, its beauty, and its philosophical implications.

The Fibonacci spiral is still used today; people use it as a guide to align elements to create pleasing images, even in games.

Most of the time, it’s simplified into the rule of thirds, which can be easily applied when photographing, filming, or composing. You just need to align the most important element of the image with one of the guides.

Another tool is a dynamic symmetry grid. Aligning certain elements with its lines and intersection points on the grid can make a complex composition feel balanced, structured, and visually appealing.

There’s also the golden triangle, which is based on the golden ratio. It helps with diagonal compositions.

Frames within frames

Another way to add emphasis to an element within your shot is to use a frame within a frame. Here we can see the character framed by the magical spheres, making her appear grand and epic.

This can also be a handy tool for more than just characters. For example, in this image, the window acts as a frame for the statue. It feels orderly, provides a nice rim light, and makes the silhouette stand out clearly.

In this example, the character in the distance is framed by the walls of the tunnel. The light pooling behind her is making her silhouette stand out very clearly, giving the player narrative direction. She would probably be invisible or quite obstructed if she were positioned anywhere else. Element placement in scenes is very important to take into account!

Adding texture

When done correctly, texture added to an image can evoke other senses. For example, a skilled classical painter can make fabric look so real that we almost know what it would be like to touch it. We all know the sensation of grass slipping through our fingers, splashing in a puddle, or feeling the warmth of a campfire. Seeing images of these experiences can unconsciously bring back memories of when we felt them ourselves, creating a more engaging and immersive experience.

The trailer of the 4th Witcher game starts with a series of images just like this, evoking a range of feelings. First flowers floating in water; we can picture how these would feel to touch, can imagine the cool water.

Then a man sitting by the glow of a campfire, feeling it with his hand. Most people will have sat by a fire at least once in their lives, and have felt that same warmth.

And then a lady being washed. We can imagine the softness and warmth of the cloth; perhaps also a chill in the air.

Meanwhile, we hear the sounds of water and a crackling fire. These textures bring us into the world, give us a sense of what it would be like to be there in the room. It’s a subtle way of worldbuilding that doesn’t require any words. Texture can be just visual, but it’s even better when it’s accompanied by excellent sound design.

Cyberpunk 2077 uses texture but in a techy, futuristic, and psychedelic way. It similarly sells you on the world, even though elements of it are not relatable. We can still get a better sense of what it would be like to experience it. 

This establishing shot immediately conveys how grimy, smelly, polluted and overextended the world is. It doesn’t just tell you the city has a polluted junkyard – it shows it, with rich textures that give you a sense of what it feels and smells like, just from looking.

Here we see a completely different side of the city – way more glamorous. Water adds a strong textural element, feeling cool and fresh. Combined with that crisp sunrise over the city skyline, it creates a beautiful scene that emphasizes the feeling of a cold morning.

In this image, we can see what happens when our character is damaged: a red overlay appears, as if the tech in his head is glitching. Using an overlay like this for a first-person character ������’t new, but it’s an effective, immersive way to signal that the player is close to dying. It mimics the loss of vision someone might get from a serious hit. Here, the glitch fits perfectly with the game’s cyberpunk aesthetic. By establishing a clear setting and reinforcing it with textures wherever possible, you give your game a distinct, recognizable feel — even in a single frame.

This shot from the trailer feels simultaneously like an old-school computer glitch and a psychedelic experience. It’s a great example of a distinctive, textured effect that perfectly suits the game’s setting.

Texture doesn’t always have to be realistic or something we have felt before. In some cases, you can get away with creating a completely new style of textures, and people still relate to it. Minecraft is a great example of this – you can almost touch it, like it’s made of LEGO. Every block has a sound effect, and it has such a distinct feeling. Because of this, it has become completely iconic and timeless.

A big amplifier of texture is lighting. It sets the tone – selling a scene as a cold, wet morning, or a hot summer day – and it makes highlights sparkle. It brings out the lushness in foliage and the shapes of your landscape.

Things like lens flares, mist, god rays and particles can add texture – layers of interest and shine – to your images in subtle ways. If you imagine the following image without the light filtering through the waves, without the godrays and particles, it would be a dryer, less impressive image.

Lens effects, like bokeh and bloom, will also help you get to where you need to be. In this case, the horizontal flare is a lens effect, and the eyes are super bloomy as well. Bloom is a post-processing effect that adds a bright, hazy glow around a light source, creating drama and intensity that wouldn’t otherwise be there.

Bokeh is what happens when you use a narrow depth of field and the blurry background light begins to form shapes and blurs together in an aesthetic way. It's not just the blur itself, but the pleasing, soft, and often circular quality that makes a subject stand out. Good bokeh can dissolve distracting clutter, add emotional depth, and transform out-of-focus lights into attractive, creamy tones or glowing highlights. 

As long as photography has existed, people have tried doing all sorts of things to achieve cool effects. They put different kinds of filters on their lenses, they hold up prisms or glass cups and photograph through them. They add a shape to the lens to create a distinct shape in the bokeh. Sometimes people apply Vaseline to a filter to place it on the lens, this gives a soft, dreamy effect. 

People like collecting old, strange cameras that produce fun and unexpected images. Others prefer to shoot on film and use techniques to alter the results at the developing stages. There are loads of possibilities, yielding unique and sometimes very beautiful-looking results. The true fun lies in the unpredictability of the final image.

Director Bertrand Mandico, who likes to be experimental with his imagery in his films, used a variety of filters on his cameras in his film, “”. He doesn’t like to rely on post-production, so he worked together with his director of photography, Pascale Granel, to create these dreamy-looking images straight out of the camera, giving their movie a distinct look and texture.

When we look at digital art created in a game engine, lens effects can be mimicked by post-processing effects. They are usually used in moderation because they can interfere with the playing experience. That doesn’t mean that experimental games don’t exist. Dujanah, for example, is a strange-looking Claymation game that has incredible texture and personality.

“Lens effects” or post-processing effects that look like lens effects sometimes get used in mainstream games. Here, Geralt from The Witcher has drunk a little too much. As a result, he waddles around, and the player has to squint through an intense-looking scene.

Finally, we come to patterns and texture. Patterns are visually satisfying and very recognizable.  A great example is Stardew Valley, where part of the fun in designing your farm is creating these visual patterns with your crops. The designers used tiling textures for elements like paths and ground, which creates a very cohesive and pleasing aesthetic throughout the game world.

There are countless ways to add texture to your experience, and it’s a good idea to keep an eye out for opportunities to enhance it. It just makes for a better, more immersive and memorable experience.

Going for the unexpected

Oftentimes, being creative and doing something weird is just more fun! It surprises the viewer and keeps them on their toes. 

In this case, they could have just filmed the shooter, but showing him reflected in the glasses is a far cooler shot. This choice is highly impactful as it implies we are seeing the last image the victim saw before the bullet hit.

Another example of a creative shot comes from Arcane, where the main character is talking to herself in a broken mirror during a psychotic episode. The fractured mirror immediately sold us on her broken state of mind. The symbolism was lost on no one. It's creative, looks compelling, has great texture, and, most importantly, drives the story forward.

It was also interesting to see the motif come back in a slightly different form with another character.

To conclude…

There are countless ways to craft a compelling composition, with plenty of tricks you can mix and match to tell your own creative stories. In general, it’s a good idea to be mindful and take a moment to consider how you place your elements and what you add to them. Do the main elements stand out? Are they arranged harmoniously? Could more texture enhance the scene, or bring it closer to the overall mood? Or maybe there’s room to do something unexpected!

I hope this article has given you a clearer sense of the building blocks behind great composition — and some fresh ideas to try in your next project.

Thanks for reading until the end!

All trademarks and copyrighted imagery are the property of their respective owners and are used here for educational and analytical purposes.

Alessio is a talented software engineer in love with new technologies, innovation, and research. He moved from Italy to the UK in 2015 to complete his PHD at the University of Hertfordshire, then joined �㽶��Ƶ in 2021. He has been with us for four years now, and continues to wow the team every day with his skills, especially when it comes to Gaussian Splats! 

Tell us about your journey and how you got into software engineering! 

Well, I started using computers for 3D animations and writing code from a very young age, and my passion just never extinguished! 

I studied computer science and graduated from Università di Catania in Italy. During my time there, I competed worldwide at Microsoft’s Imagine Cup in 2013 with my Italian team, “TeamNameException”. We even won first place!

In 2015, I  moved from Sicily to the UK to study at the University of Hertfordshire. That’s where I completed my PhD and published my thesis on ”.

After my PhD, I joined , a company specializing in real-time camera tracking for films and live events. There, I expanded my skills in C++ and Unreal Engine while working on challenging AR pipelines to previsualize movies and live events – including the !

In 2021, I joined REWIND (now �㽶��Ƶ), and the journey has been incredible ever since.

What was it that attracted you to �㽶��Ƶ?

I originally joined REWIND before it was acquired by �㽶��Ƶ. I was fascinated by the talent in the company and its great portfolio of VR and AR experiences. As mentioned, I have always been passionate about XR throughout my career journey, and joining REWIND felt like a natural continuation of that passion.

Since REWIND became �㽶��Ƶ, I’ve been even more excited to see how much bigger our reality has become, with colleagues in both the UK and the USA, and with many global projects, ranging from virtual production to XR apps and games.

The culture and the friendly, knowledgeable people that make up our teams are unbeatable. We have lots of opportunities to mentor others and be mentored ourselves. This not only sharpens our skills but also makes learning a key driving force behind our powerful team. I am honored to be part of it!

Tell us more about your current role at �㽶��Ƶ!

Currently, I am working on our OKO platform as a Senior Specialist Engineer, jumping between Unity and Unreal to improve our implementation and add new features. The fun part for me is to study and integrate brand-new, cutting-edge technologies into our products, especially in the field of digital twins, XR, and radiance fields.

What has been your proudest/standout moment while working with us?

One of my proudest moments has been the success of our core product, OKO, after all the efforts my colleagues and I have put into it. While working on OKO, I got the opportunity to study and integrate new technologies (such as Gaussian Splats), work with a range of different platforms, and expand my knowledge and skills in the process.

Making the Gaussian Splat viewer from scratch for our OKO Unreal plugin and OKO Unity implementation has definitely been another fantastic personal achievement, together with some recent hackathons and proof of concepts I have been involved with.

Last year, I also got the chance to showcase OKO and our team work on stage at the, which was a very rewarding and exciting experience.

What is the biggest lesson you’ve learnt in your career?

Never stop learning and enjoy knowledge sharing with your colleagues. There is always something new to study, research, and turn into innovative solutions. That mindset fuels my passion for innovation, making it nearly impossible to get bored with my job. Throughout my career, my role has always involved challenging engineering problems, rarely explored by other companies. Instead of seeing them as a burden, I see them as a chance to experiment with new things, making them more fun and entertaining. 

Here at �㽶��Ƶ, our learning is stimulated even more by the number of mentoring sessions, hackathons, and dedicated R&D spikes that we perform every sprint. I really value this aspect of my career, not only because it keeps me motivated, but also because it gives me the opportunity to create things that haven’t been explored before by the industry. An important lesson I’ve learned is that a team grows together. By building strong relationships with colleagues, tackling challenging problems collaboratively, and being proactive in the interest of our projects, the whole team thrives – and everyone’s success naturally grows along with it.

If you could wake up in the body of another person (just for one day) who would it be and why?

If fictional, I would choose Dr. Emmett Brown from Back To The Future, so that for one day I could have his creative genius mind to make crazy inventions, travel in time, and drive the legendary DeLorean.

If I could choose to go back to the 90s, I would say Steven Spielberg, while shooting the first Jurassic Park – he is just a master filmmaker, plus I would love to see backstage!

What are you reading/listening to right now?

I recently read . It’s striking how relevant the themes of surveillance and control still feel today. A recommended read for sure.

Tell us something that would surprise us!

I am almost a pro at playing with the Quest 3! I started during the lockdown, and got quite competitive at it 🙂. Another thing that might be surprising is that apart from coding, I love making 3D animations! And when I’m inspired, I publish them on my where I also have some video tutorials and showcase . Recently, I also made an for my hobbyistic 3D modelling journey. It’s something that relaxes me and helps me to express my creative side.

If you could have a superpower, what would it be?

I’m torn between choosing teleportation or the ability to know the future. But I’d probably go with teleportation — it would save so much time and money, and open doors to places you’d never normally reach (yes, even bank vaults!). Plus, life’s more exciting when the future still holds surprises.

What is something you think everyone should do at least once in their lives?

Visit Sicily and have an almond & chocolate granita with a brioche while staring at the sea in Catania; you will not regret it!

Apart from that, make your dreams come true and stop postponing, we only live once – do not waste it. 🙂