Blog

A few months back, a VC firm reached out to me, inviting us to join their Spring 2024 program. I quickly brought a couple of friends into the fold as co-founders. Together, we applied for MiraclePlus, led by Lu Qi, which you might (but probably not) know as the former Y Combinator China. I’m working on an upcoming blog post about our entire journey. Spoiler alert: we made it to the final interview round! For the time being, I'm here to share our application with you (but I'll keep my co-founders' details under wraps).

Company name Cubic Inc.

Company URL, if any
10 Millions 3D Models and a Cubic World

Product name
Cubic

If you have a demo, what's the URL? For non-software, demo can be a video. cubic.mov
(… Can’t share with y’all at this moment, sorry!)

Describe your product in 50 characters or less.
Generative AI, datasets, and tools for 3D

What is your company going to make?
Cubic’s goal is to enable Generative AI for 3D. We achieve this by 1) Creating a photogrammetry tool set and APIs that allows companies to scan objects to get 3D models. This process could be automated too. People can download their 3D models for free if they allow us to upload them to our platform. 2) Create a platform that allows people to open source their models — like GitHub for 3D. 3) Train generative AI based on our data. A GAI for 3D that enables possibilities of creating tailored made experiences to each individual user on scale unheard of. We outlined this vision and philosophy in our URL.

What categories best apply to your company?
B2B, Community, Artificial Intelligence, Developer Tools, Augmented Reality, Virtual Reality, Open Source

Which city are you currently living and working in?
4

What is the current career stage of the main founder?
Student

Please record an 1-minute video introducing the founder(s), and briefly describe the startup project. This video is an important part of the application.
(For privacy reasons won’t share here. But know we are all little embarrassed.)

How long have the founders known one another and how did you meet? Have any of the founders not met in person?
We known each other for more than 3 years. We are colleagues and good friends in and outside of the lab. All of us met each other before.

Name
Yuhao Chen

Email
hello@chenyuhao.me

Phone
(No.)

WeChat ID
(Also No.)

Date of Birth
(No.. But I am 23 as of now, and born on New Year eve. You do the math.)

Hometown
Korla, Xinjiang

Educational Background - Schools, Degrees (including field of study), and Years of Graduation
University of Southern California — M.S. Computer Science: Game Development

• Class of 2023 University of California — B.A. AGPM
• Class of 2025

Work Experience - Employers, Position / Titles and Dates
UCSC AVIX Research Lab

• Project Lead, Researcher
• 2021 - Present
• Enhanced Photogrammetry with single-view or few-views - FLAME fitting and D3DFACS
• Deep Learning (specialized in avatars generation) Qing Mang | Readland
• Web Developer, Software Engineer - 2023 - Present
• Working on an AI wrapper apps
• Full-stack web and iOS developer

Please tell us about the time you most successfully hacked some (non- computer) system to your advantage.
I hacked my mindset. Increasingly I am realizing the importance of long term thinking and the value of scarcity. In college, people tell you bullshit like don’t read, don’t learn anything deep, don’t be a geek. Pick computer science and learn web dev so you are set for life. But if you think for the long term the only thing that matters is learning as much as you can. Fail as quickly as you can so you learn from mistakes earlier than others (and when risks are lower). Don’t get bogged down by not finding success now. And if you can offer scarcity in an age of abundance. Success finds you. You’d be surprised how many systems you can hack with this mindset. Coming from a rural place meant I needed to walk faster than everyone else. So far, this is my secret sauce.

Please tell us in one or two sentences about the most impressive thing other than this startup that you have built or achieved.
I launched seven quality apps on App Store in the span of a year. Won an award, got many five stars reviews, and more than 10K downloads.

Are you a technical founder? Y/N – Yes

Do you have at least 5% equity in the company you're applying with? Y/N – Yes

Will you come to Beijing to participate in the office hours and events hosted by MiraclePlus Mar through Jun 2024, if we fund you? Y/N – Yes

What is the company's progress for the past 6 months (in terms of tech development, user acquisition, revenue and funding)
Our project borned from a research lab. For the past 6 months, the lab has launched our Photogrammetry app for desktop. Our iPhone version is ready to release. We have some breakthroughs in single-view facial 3D generations via FLAME fitting. We are also actively working on GAI technologies like IADB and other diffusers.

How long have each of you been working on this? How much of that has been full- time startup? Please explain.
So we worked on photogrammetry and 3D GAI tech for about 2 years at this point. But the AVIS lab we are working for are doing this for much much longer. The Phd and the professor who are directing us have many years experience in this field. For us we haven’t founded a company before. But our Professor James Davis, previously co-founded a company around 3D facial scan, and was eventually sold to Meta (Facebook at the time).

Are people using your product? Y/N – Yes

When will you have a prototype or beta? If it's already live, when was it launched?
Like we said, we already have a macOS photogrammetry app launched. The iOS version is ready to launch. The web tool set and api we are proposing could have a prototype in a few weeks. You can download our app here: https://apps.apple.com/us/app/cubic-photogrammetry/id1632546962?mt=12

Do you have revenue? Y/N – No.

Which group of people are your customers? What problems do you solve for your customers?
1) Companies from virtual reality sector. With the Oculus and launch of Vision Pro, we anticipate growth and new compaines in this area; we would like capture some of that. 2) Game studios. We believe they would be interested in both 3D model dataset and our Generative AI for 3D products. 3) We anticipate our biggest customer early on are other companies working on Generative AI. Our photogrammetry toolsets are extremely useful for generating large amount data for training.

Why did you pick this idea to work on? Do you have domain expertise in this area? How do you know people need what you're making?
So this idea came from our own need at research labs. In research, we worked on many generative AI projects. And the biggest problem are almost always data. Our team is on the young side, but we have a lot of expertise in 3D, generative AI, and photogrammetry. We are also experienced in building and scaling web and mobile applications. I also want to mention, Professor James Davis at AVIS lab, is actively supporting and behind us. He had previously started a successful company in this field.

What's new about what you're making? What substitutes do people resort to because it doesn't exist yet (or they don't know about it)?
We want to be an integrated and comprehensive tech providers in the field of generative AI for 3D. In past, companies and communities relied on sourcing solution from all sorts of companies. Photogrammetry, for example, relied on companies like Agisoft Metashape and 3DF Zephyr for 3D reconstruction. Their software are highly inflexible, difficult to operate, and high license fee. We want to offer better product for almost free because we understand 3D training data for AI are very valuable. This makes us a disruptive force in 3D reconstruction tech. On AI side, 3D data are incredibly difficult to find. Some of the best AI models for 3D are trained on only thousands of models. This because 3D models are incredibly expensive to make. Companies and teams sometime relied on open source dataset of inconsistent quality. Or pays millions of dollars to purchase models. This field is very much overlooked. If you go to hugging face right now, there are literally only 2 datasets for 3D. https://huggingface.co/datasets? task_categories=task_categories:image-to-3d&sort=trending

Who are your competitors, and who might become competitors? Who do you fear most?
Like I said Agisoft Metashape and other companies that provide 3D reconstruction are our direct competitors. We also worry about game companies, they have a big library of 3d models can be used for training. However, we are not particularly worried about them. I believe Microsoft is a company we should fear. They have dozens of game studios. Partnership with OpenAI, and access to computes.

What do you understand about your business that other companies in it just don't get?
We understand that data are supremely important to generative AI. We understand 3D for AI will have a big breakthrough if only someone have enough training data. Lastly, we understand how to get those data.

I’ll begin with a paraphrase of a Ben Thompson article, DALL-E, the Metaverse, and Zero Marginal Content.

In the past two decades, social networking has created some of the most successful companies. They succeeded because they effectively served user-generated content (UGC). Facebook and Instagram, for example, use an algorithmic timeline to surface the best content from your network. TikTok, however, serves content from across the entire network.

The reason UGCs are so lucrative is two-fold: 1) they can be tailored to the interests of individual users, and 2) they are essentially free for the companies, with the real challenge being delivering the right content to the right audience.

Media products like games, however, are the opposite. They are expensive to make, so 1) AAA games need to target a broad audience, and 2) players come to the game if it matches their interests.

This is also why most social networking companies are like rocket-ships – they serve everyone if they are a success. And game companies are like tractors – every penny matters. (Ben Evans)

With generative AI, we have an opportunity to deliver on the promises of virtual reality (or the metaverse, if you must) that gaming couldn’t deliver. We can drive the costs of 3D models down to zero with generative AI.

I will define an arbitrary goal here: 10 million 3D models are what stand between us and the virtual/augmented reality we've long been tantalized by.

In the meanwhile, I think we have a cost-effective way to get there. We can leverage existing technologies that automate 3D creation (e.g., Photogrammetry) and acquire models that are already made. It would still be a monumental task, but the results will be well worth it —

A foundational model that generates a sprawling, immersive digital world for each individual user.

Cubic World

Many people harbor reservations about a world rooted in virtual reality, conjuring images akin to those described in Ray Bradbury’s dystopian works, Fahrenheit 451 and The Veldt. These concerns are amplified by our world's increasing digitalization, fueling apprehensions about losing touch with the tangible, human elements of existence.

Yet, this perspective only captures one side of the story. A closer look reveals a world where people, now more than ever, value physicality and authentic experiences. This is evident in the resurgence of vinyl record sales and printed books, and in the renewed success of physical storefronts. These trends suggest a deep-rooted human desire to connect with the physical world, highlighting an intrinsic appreciation for tangible experiences.

This dichotomy indicates our world is splitting into two realms: the real world, our blue sphere, a place of breath, birth, and life's tangible joys; and what I call the cubic world, a realm of immersion, tailored to individual preferences and under personal control.

The cubic world, rather than being a dystopian escape, should be envisioned as a complementary space. It offers a canvas for creativity, exploration, and personalized experiences, expanding our reality rather than replacing it. In this cubic world, the boundaries of physics and geography dissolve, enabling us to transcend the limitations of our physical existence.

A few months ago, I finished reading Matthew Ball's Metaverse and jotted down some thoughts on the topic in a blog post. However, I stopped working on the blog that post was on shortly after. I felt like I had few things I wanted to say in that post, but my ideas were not well articulated. Now, I am revisiting and refining them.

The ‘metaverse’ has gained significant attention from players in the gaming space. Partly because they see it as a chance to prove their worth and compete with their tech counterparts. The gaming industry, with a projected revenue of US $250 billion. Yet, this number is dwarfed by the trillions generated by the tech industry. While this comparison is flawed, it highlights why many in the gaming space hoped the ‘metaverse’ as their big break to take on the internet and tech, and that they need to seize the moment.

So, if we imagine a future where the ‘metaverse’ does become an exciting addition, substitution, or evolution to today's internet. Will gaming emerge victorious over tech in this process?

I have serious doubts.

From a product perspective, the gaming industry has released many unfinished games recently, often resulting in launch day disasters. Take No Man's Sky, Fallout 76, Anthem, or Cyberpunk 2077, just to name a few. On the other hand, their tech counterparts have shown more consistency. For example, when Threads was launched, it gained a record-breaking 40 million sign-ups on the first day. Despite some users reporting crash issues and glitches, the launch went mostly smoothly. Sure, Mr. Musk is burning down X as we speak. But that one is intentional.

Don't misunderstand; the majority of games are excellent and launch without a hitch. However, the issue of inconsistency in the gaming industry is real and has undermined consumer confidence. It also highlights the distinct challenges faced by the gaming industry compared to tech. Tech's success is partly due to its disruptive nature. Initially, it offers products or services that may provide the same or slightly inferior experiences compared to existing ones, but at a fraction of the price. This strategy attracts price-sensitive consumers who don't value the additional services provided by incumbents. Over time, technological advancements enable disruptors to improve both their products and operational efficiency, eventually winning over the entire market.

Yet, none of this applies to gaming. The costs for AAA game development have skyrocketed despite technological advancements. The absence of efficiency gain can be partly explained by a combination of inflation and the stagnant pricing of games. But the primary driver is the cost of art assets, such as 3D models, textures, animation, rigging, and sound. As technology allows studios to aim for ever more ambitious projects, the cost of creating these intricate art assets does not scale proportionately, leading to ever-rising development expenses.

To understand the magnitude of these costs, consider the example of a basic 3D character for a simple student capstone project. If students were compensated for their work, the creation of just this one character could amount to around $1720.

1 conceptual artist          x  5 hours
1 modler                     x 15 hours
1 texture artist             x 10 hours
1 animator/rigger            x 10 hours
---------------------------------------
Total: 40 hours

This high-cost structure has led publishers and studios to outsource art creation, often to countries with lower rates, raising ethical concerns about unfair treatments.

The gaming industry's hit-driven nature, despite the high development costs, actually works to the advantage of large studios and publishers. Joost Van Dreunen's observation in "One Up" highlights this dynamic:

”Strongly hit-driven success is another defining aspect of the games business. A popular title will generate disproportionally more revenue than its immediate peers... In 2018, FIFA 18 was the top-grossing title in the sports category on console with $723 million out of more than $2 billion for the entire top ten, or 28 percent… Succinctly put, for every blowout success, dozens of titles fail to deliver on expectations. The winner-takes-most aspect of game publishing means that an organization is either doing really well or dying."

This meant established publishers like EA, which boast a portfolio of hit series, benefit from stable revenues. This financial stability allows them to reinvest comfortably in both in-house and outside studios to create the next hits, bearing relatively low risks. This creates a significant competitive moat and provides these publishers with considerable leverage over other studios.

This also meant players have diminished leverage against big gaming studios. So it doesn’t really matter when a studio fucked up their post-apocalyptic MMORPG. You still gonna play their Sci-Fi RPG because they are the few ones who have deep enough pocket to make a game at that scale.

However, this very nature of the gaming industry might be its Achilles' heel in a face-off with tech. Companies with big moats, often lose sight of the need for operational efficiency, leaving them vulnerable to disruption. This places the gaming industry at a potential disadvantage, challenging its ability to compete effectively with its tech counterparts in this new digital frontier.

And a disruptive technology might already be here. Generative AI like Midjourney, DALL-E, and Stable Diffusion have been proven to be able to do remarkable things despite poor taste. However, exploring the full implications of generative AI for games is a vast topic in itself, and should be left for another day.

I didn’t buy the ‘metaverse’ hype a little over two years ago. And I certainly don’t buy it now. But I think it’s a topic that is set to come back in due time. And it’s easier to write about it when the heat is off.

It is important to note that the content of this document is subject to constant change, so it is recommended to refer to this document regularly when writing.

highlight.js does not support syntax for Yarn Scripts. So syntax highlight on for this page will be wrong.

Events

In Brinkmanship, we defined a special type of nodes –– Events. They are nodes that trigger when specific events happen in Unity. They act as starting points for their own series of dialogues, similar to the start node. An event will always named with a keyword "event" prefix.

In general, it is not recommended to jump to events from within other nodes in Yarn Spinner. This is because jumping to an event will bypass the associated logic that is usually triggered when Unity starts the event. However, if you wish to safely trigger events from the Yarn Spinner, it is possible to develop a method. Please inform the engineering team if want this.

When an event is triggered, the game will remember the node that led to it. This is so the game's where to resume dialogue when this event finished (see command <<resume>> and <<resume_stopped_node>> for more details). However, if a node's name contains the keyword "event", the game will not remember it. Therefore, avoid naming nodes with "event" if you want them to be remembered.

Events will always interrupt the current node and restart from themselves. If you need commands to disable and re-enable trigger detection for a specific event, please let us know.

Typically, a single event is consist of multiple related sub-events. Functions and variables are usually provided to separate them. A good practice is to create a node for each sub-event, and name them on_name_event, where name is the name of the sub event. The keyword "event" at the end will prevent the game from auto resume from this point.

Brinkmanship DialogueManager requires all events listed here exist in a project.

event-npc-death

Will trigger when a npc dies in the scene. This event should be used with is_all_npc_dead() and is_npc_dead(name : string) functions to determine which npc is killed.

// EXAMPLE
title: event-npc-death
---
<<if is_all_npc_dead()>>
	<<jump on_captain_pilot_both_died_event>>
<<elseif is_npc_dead("Pilot")>>
	<<jump on_pilot_killed_event>>
<<endif>>
===

event-gun-threat

An events that triggers when player tries to threaten npc(s) with their weapon. Please note, this event can be triggers when the player either raised, aimed, or holstered their weapon, or fired the weapon but missed, no ammo, or hit an object that can’t be killed (like a door). This event doesn’t trigger when the player hits a npc. Because that kills them and should be handled by event-npc-death. To separate what type of threat was made (or disengaged), use get_variable(var : string) function with variable gun-threat-state

// EXAMPLE
title: event-gun-threat
---
<<if get_variable("gun-threat-state") == "holstered">>
	<<jump on_gun_stow_event>>
<<endif>>
===

event-resume

A special event triggered by <<resume>> command. This event is for adding logics to redirect conversation back to the correct node. However, you do not have to cover all resume scenarios. Command <<resume_stopped_node>> can automatically find the last node player stopped at before an event was triggered. Which is why you should always place a <<resumestoppednode>> at the end of this event to serve as a fallback.

// EXAMPLE
title: event-resume
---
<<if is_all_npc_dead()>>
    <<jump on_captain_pilot_both_died_event>>
<<else>>
    <<resume_stopped_node>> // This command should always-
	// be placed at end of this nodeto catch scenarios-
	// that's not covered by logics above and provide auto-
	// resume as fallback.
<<endif>>
<<resume_stopped_node>>
===

Functions

In Yarn Spinner, functions are units of code that Yarn Spinner scripts can call to receive a value. Functions don’t always take parameters, but will always return a value. Yarn functions can return the following types of values:

• string
• int
• float
• bool

Functions can be used with events to separate them into more detailed sub-events. And can be used in any other nodes as well.

is_all_npc_dead() -> bool

Returns a bool. When all npc in scene is killed, this function will return true. This function doesn’t take any parameters.

//EXAMPLE
<<if is_all_npc_dead()>>
    <<jump on_captain_pilot_both_died_event>>
<<endif>>

is_npc_dead(name : string) -> bool

Returns a bool. Return true when the passed in npc is dead. To check if an npc is dead, pass in parameter name as string,

//EXAMPLE
<<if is_npc_dead("Pilot")>>
	<<jump on_pilot_killed_event>>
<<endif>>

get_variable(var : string) -> any

Can return either string, int, float, and bool. Takes a parameter var as string. This function retrieve data stored in game’s storage system. See variables section. A set_variable command can be implemented, please let us know if you need it.

Commands

Yarn commands allow you to trigger codes (functions) in Unity side from YS. Commands can take parameters but never returns. You use commands similar to how you use <<jump>> in YS. It is possible to add commands that trigger all kinds of things in Unity. Such as trigger an animation or walk a npc to a location. Please let us know if you need a specific commands.

<<resume>>

Jumps to event-resume. You may wonder why don’t just use <<jump event-resume>> Using <<resume>> will make your script more readable. And give us options to write Unity side logics that go with <<resume>>

<<resume_stopped_node>>

Jump to last stopped node that got interrupted by an event. The game will never resume to a node that contains keyword “event” in it.

Variables

Brinkmanship variables are used to retrieve value from Unity side through get_variable(var : string) -> any function. And should not be confused with yarn spinner’s built in variable. Right now only one variable exists, but almost all variables from Unity can be linked with YS in this way. Informations like ammo count or number of times raising the gun can all be variables. So let us know.

gun-threat-state

This variable returns a string that contains the state of gun threats. The returned states track following states: holstered, raised, aimed, firing-no-ammo, hit, missed, and hit-attackable-disabled. And with the last one meaning gun fired but hit a npc that can’t be killed (like a door). gun-threat-state should almost always used in the context of event-gun-threat.

Part I

Josh Lyman, or perhaps C.J. Cregg from The West Wing, once quoted Mark Twain in a letter: "I didn't have time to write a short letter, so I wrote a long one instead." "This is going to be a very long letter," he wrote. And indeed, this will be quite a lengthy one.

Someone once asked CGP Grey about "letters to my future self." He responded, "I doubt my future self would care much about a letter from my present self." I can relate; even my thoughts from three days ago feel uninspiring, let alone a decade later. I've attempted blogging twice before and tried journaling five times, yet they all felt like notes to a distant version of myself.

Part II

“Writing is something that needs to be learned; writing shouldn’t be exclusive ability of certain professions. Apart from the very few who wrote for the public, we should write for ourselves. This is a record of our own life, and it is also a way to expand ideas and ways of thinking.”

Quoting Ted Chiang, it is Zhao Poe @bluebird0605 that reminded me that writing shouldn't be reserved for specific professions. It's a means to record our lives and expand our perspectives. Though I'm not familiar with Ted Chiang's work (wasn't a fan of the film Arrival), I find Zhao Poe and Ted Chiang's argument quite compelling.

Part III

Here's my plan for this place: 1) I'll write some lengthy pieces. 2) I'll also jot down bits of thoughts here and there. This is how Li Ruyi from Yitianshijie does it; how John Grubber for Daring Fireball does it, and so can I.

Haruki Murakami once shared his writing routine: reading, listening to records, and then producing exactly ten pages daily—no less, but importantly, no more. While engrossed in A Wild Sheep Chase, Enter the Sheep Professor, I abruptly paused, grabbed my notes, and found myself here. I quote to preserve what I've absorbed and write to expand upon my thoughts.

– Yuhao