Open source UE5 Illusion-like studio, scene creator, character creator, game creator, animation and pose library

[Microtom]

Goal

The goal is to create an an open source unreal engine version of an illusion-like studio, like koikatsu and honey select. The current illusion studios don't allow the easy import of props and aren't open source.

UE being a game engine, the studio won't simply allow to create images or videos for visual novels made in renpy. The studio itself will allow the creation of the visual novel, as all as allow characters to be played with a controller. This gives a lot of additional opportunities.

This project is ambitious with nothing to show for it, but I want to mention that with the arrival of power AIs, learning development has been greatly facilitated. Free learning resources are also widely available. Everything but actual labor is present to achieve success.


Project Governance

If the project is open source and people are interested in participating in it, the project would need a system of governance. Asking chatGPT about options, it suggests multiple.

Spoiler: ChatGPT Systems of Governance

You don't have permission to view the spoiler content. Log in or register now.

The choice of a system of governance will have to be discussed.


Version Control

The project will need to use a version control system(VCS) to allow collaborative work. Such a system is well implemented in UE. According to ChatGPT, there are multiple options other than GIT.

Spoiler: ChatGPT VCS options

You don't have permission to view the spoiler content. Log in or register now.

The choice of the project VCS can be discussed, but GIT is an adequate option. Here is a tutorial on its setup:

You must be registered to see the links

Feature List

To be determined excatly, but the feature list is essentially what the Illusion studios already contain.

Character creator: A character creator will require a single mesh with multiple morph targets. Characters that already exist, such as daz' genesis 8, can be imported into UE, but the mesh isn't optimized for real time game engine.

Animation set: AI tools such as

You must be registered to see the links

allows the creation of animations from simple videos and more AI tools will created. The resulting animations can then be corrected manually if needed.

Style

The illusion studios have only one style. This isn't a limitation that's applicable here, we can have as many styles as we want. UE has large libraries of assets, such as realistic asset from Quixel, so the studio can have a realistic style. Personally, I want to work on a drawn style that I can easily reproduce using Stable Diffusion. It's derived from Darkcookie's Summertime Saga. Here's a sample:

Spoiler: Summertime Saga Style

You don't have permission to view the spoiler content. Log in or register now.

Tutorial

GIT setup:

You must be registered to see the links

 

[dabuty]

I have read your proposal and it looks to me that you want to make something that might be already done.
You cant simply make a proprietary software opensource. You cant make unreal engine opensouce, not legally posible.
What is your purpose on doing a huge task of building an engine? Are the engines availble not up to the task?
Are you going to build a game with the engine or just the engine?

Im not sure if you are aware of already existing tools that are open source:
Godot is a an open source game engine that recently release the version 4 with much powerful 3D graphics than before
MakeHuman is a open source human generator that has existed for at least a decade now.

 

[Microtom]

I have read your proposal and it looks to me that you want to make something that might be already done.
You cant simply make a proprietary software opensource. You cant make unreal engine opensouce, not legally posible.
What is your purpose on doing a huge task of building an engine? Are the engines availble not up to the task?
Are you going to build a game with the engine or just the engine?

Im not sure if you are aware of already existing tools that are open source:
Godot is a an open source game engine that recently release the version 4 with much powerful 3D graphics than before
MakeHuman is a open source human generator that has existed for at least a decade now.

You misunderstood. UE is already a game engine. What I want to do is a studio at a higher abstraction level similar to the Illusion studios like koikatsu. There would be a GUI to easily select pre-made characters, animations from libraries. You mustn't know what the Illusion studios are.

 

[Lerd0]

 

[Winterfire]

This whole thing doesn't make any sense. Why would you make a smaller wheel out of an already existing wheel?
Everyone can download Unity or Unreal right now, for free. Both have a scene editor, which is what charastudio is.
Create the scene you want in Unity or Unreal using all the plugins and stuff you want, bake the lightmap (which is an option that doesn't exist in charastudio), place the camera wherever you want, and grab a screenshot. Done.
Why the extra iterations and steps?

 

[dabuty]

You misunderstood. UE is already a game engine. What I want to do is a studio at a higher abstraction level similar to the Illusion studios like koikatsu. There would be a GUI to easily select pre-made characters, animations from libraries. You mustn't know what the Illusion studios are.

Yes I dont understand what you want to make. I dont know what Ilusion studios are or what they make. If you could explain it it would be great.
I know that Unreal Engine is a game engine, I have used it. They have the metahuman character creator and they have a lot of animation built in.
But I dont know what problem you are trying to solve. So I dont knwo if I can help you

 

[Microtom]

Why would you make a smaller wheel out of an already existing wheel?
Everyone can download Unity or Unreal right now, for free.

The Illusion studios are made in Unity. People still use them instead of using Unity because the level of abstraction is higher and the studio contains characters creators and already made assets, they are easier to use.

The project isn't redoing UE, but it's also not just a set of assets to use within UE.

 

[Microtom]

Yes I dont understand what you want to make. I dont know what Ilusion studios are or what they make. If you could explain it it would be great.
I know that Unreal Engine is a game engine, I have used it. They have the metahuman character creator and they have a lot of animation built in.
But I dont know what problem you are trying to solve. So I dont knwo if I can help you

Many of the visual novels here are made from renders made in the Illusion studios. Here's a video of the Koikatsu one:

You must be registered to see the links

 

[dabuty]

Oh so its a posing tool that generates stylised 3D anime like characters.
In that case you are better off making an addon for Blender that can generate anime style 3D models .And you can use Rigify to make the skeletons and it has its own pose library built in and good animation tools.
It also has realtime non-pbr rendering.
You get so much flexibility if you go with Blender than a Game engine.
If you want to then export either the renders or the animations its easy to do for any game engine. That way people have the flexibility to develop their game with the tools they want

 

[Microtom]

Oh so its a posing tool that generates stylised 3D anime like characters.
In that case you are better off making an addon for Blender that can generate anime style 3D models .And you can use Rigify to make the skeletons and it has its own pose library built in and good animation tools.
It also has realtime non-pbr rendering.
You get so much flexibility if you go with Blender than a Game engine.
If you want to then export either the renders or the animations its easy to do for any game engine. That way people have the flexibility to develop their game with the tools they want

I want to go beyond simply posing and rendering. I want the studio to also be a tool to easily create the visual novels or games themselves. UE gives the ability to have controllable characters.

 

[Winterfire]

The Illusion studios are made in Unity. People still use them instead of using Unity because the level of abstraction is higher and the studio contains characters creators and already made assets, they are easier to use.

The project isn't redoing UE, but it's also not just a set of assets to use within UE.

Exactly, and this project nor any other is going to be offering "more". Even CM3D2 which is quite popular offers 10x less content than KK does. For what it does, KK or any other future Illusion game is more than enough.
If you want to "extend" it, simply export what you need and import it on Unity.

 

[Microtom]

Exactly, and this project nor any other is going to be offering "more". Even CM3D2 which is quite popular offers 10x less content than KK does. For what it does, KK or any other future Illusion game is more than enough.
If you want to "extend" it, simply export what you need and import it on Unity.

Illusion studios aren't game engines, though. Also, the import and export process isn't simple like it is in UE. And I wanted to use UE because that's what I'm learning right now.

 

[Winterfire]

Illusion studios aren't game engines, though. Also, the import and export process isn't simple like it is in UE. And I wanted to use UE because that's what I'm learning right now.

Yeah, they have an average scene editor. You need to choose between a vast amount of content already set up for use, or more extensitivity and tools. I get what you are suggesting "why not both?" but you have to be realistic and understand that you can't get that amount of content + create such an editor. Creating the editor itself would be a colossal task to begin with, but the content too which would require more popularity than CM3D2 and people willing to make content for it?

It is more realistic (and easy) to keep using the average scene editor, use the plugins already offered to input your custom content if needed, or if you truly need more tools or customizability, use the same plugins to export and remake your scene in Unity. There is even a project offering the shaders and all to make a very similar look in Unity, although if you were to put the effort in exporting and recreating a scene in Unity, you'd probably choose to use better shaders such as RealToon.

Not to mention that the great majority of people use Illusion only for the characters' rich content, and the various cards used to replicate their favourite characters (Often Anime), you'd also need to offer that on top of the rest.
If not, why shouldn't someone use Vroid Studio instead, which is easy to link with Unity?

 

[anne O'nymous]

Also, the import and export process isn't simple like it is in UE.

Hmm, I wonder, could it be because the meshes are copyrighted and not free to use ?


I agree with Winterfire that, at least as you present it, your project make few sense.

From what I get, you want to make an Illusion studio-like that wouldn't limit to rendering, but would also permit to export the scenes, for them to be used elsewhere. But I'm not sure that you really understand what that imply.

Anyway, I'm relatively sure that people using an Illusion studio wouldn't be interested. They choose to use it because it's an easy way to have not too bad renders even with a shitty computer. And like Winterfire said, you'll not be able to do better and more complete than Illusion that is doing this since so long now ; at least not before years.
As for those who would want a scene export ability, they don't care about Illusion studios, they use software like Blender or Daz Studio (to limits to the free options). And once again you'll need years before you can reach the level of those software that are the sum of decades of development and teams effort.

And all this for what goal exactly ? You say that Illusion studios don't allow easy import and aren't open source, but it don't prevent Honey Select studio to have tons of plugins, assets and mods. So what your own tool would add exactly ?
An easier way to do it ? What the interest if no one want to make assets for it ?
Honey Select studio have so much assets because there's the game behind, and also because Illusion is known and used for this since so many time. Daz have its store where you can sell your assets. Blender have a vast range of import filter, what mean that it don't especially need dedicated assets.
Will you own software have one, or more, of those threes, in order for people to want to dedicate time making assets for it ?

 

[Winterfire]

Yeah, I think at the end you do not realize the sheer scale of such a project.
I mean, community projects are nice and all, but there is only so much you can do especially when the usefulness is dubious and better alternatives already exist (with much less effort required by all parties).

 

[cooperdk]

Illusion style, why the hell use Unreal Engine, that bloated professional level tool?
Why not use what Illusion uses, Unity?

 

[osanaiko]

I think y'all are haters and are bringing OP down, not encouraging him.

I agree it seems like a massive project, but I can see something like what he is imagining:

"Koikatsu studio"-style asset poser that integrates directly with a basic VN builder engine where you can use images created in the poser and add dialogue and navigation choices.

Of course there are many challenges, for example obtaining non-copyrighted meshes like someone mentioned.

But the concept is cool.

 

[FromOtherSpace]

the concept is cool.

Yep, it's cool.

First, I want to say I'm all for open-source whenever it's possible. I fully support people willing to create open-source software (and arts).

But.

As said, starting from scratch is a colossal amount of work, so adding to existing projects is, in my opinion, a better choice.

Blender, a good open source software for everything 3d, got makehuman and ManuelBastoni (and a third which I don't remember its name) addons to create people. They are not perfect ; few test I have done seem less beautiful than CC or DAZ. Effort could be made to make them better and add anime shader, anatomy, etc on top on what is already done. Sidenote, for shader blender is powerful, amazing things can be achieve.

Godot 4, a good open source game engine, have a really good integration of Blender. You can create something within Blender, save it in your project (as a .blend, no export needed) and work with it in godot (I think you will need to export for game release)

So, I think it would be more efficient to work on those addons for blender to give an user friendly control on anime characters creation and, maybe, create one for godot to make an (more) easy integration to any game. For ex, they have added a skelleton thing to share libraries of animations between yours characters but there is no default animation, nor automated recognition of rig to link an asset to an existing librarie. An addon with this could be a cool thing.