AI LORAs for Wildeer's Lara Croft (Development Thread)

[Mr-Fox]

I found yet another "version" or creation of Lara, by someone named detomasso (det0mass0) on deviantart.
He created renders in daz studio (just assuming), however I have no idea if he is using one of the versions being released by "vendors" for DS or if he has created a 3d model from scratch. His gallery has ap 300 images, all of Lara. I have downloaded them all and uploaded to my gofile folder for yours and my own convenience. From this large amount I got it down to 97 images that I then edited out logos etc in photoshop. I'm doing a test run for a Lora on these images right now. I will upload these also in case anyone else is interested, though I haven't resized them only cropped a few. I'm testing to see if it's true that simply using "buckets" is enough and what the result will be.

You must be registered to see the links

You must be registered to see the links

 

[me3]

couldn't we just grab screenshots from the first tomb raider game, far less details to train and no need to worry about fingers looking weird and noone could tell if the face was mutilated when drawn at a distance

 

[felldude]

I found yet another "version" or creation of Lara, by someone named detomasso (det0mass0) on deviantart.
He created renders in daz studio (just assuming), however I have no idea if he is using one of the versions being released by "vendors" for DS or if he has created a 3d model from scratch. His gallery has ap 300 images, all of Lara. I have downloaded them all and uploaded to my gofile folder for yours and my own convenience. From this large amount I got it down to 97 images that I then edited out logos etc in photoshop. I'm doing a test run for a Lora on these images right now. I will upload these also in case anyone else is interested, though I haven't resized them only cropped a few. I'm testing to see if it's true that simply using "buckets" is enough and what the result will be.

You must be registered to see the links

You must be registered to see the links

View attachment 2929463 View attachment 2929465 View attachment 2929466
View attachment 2929464 View attachment 2929470

Could be using this

You must be registered to see the links

 

[Mr-Fox]

Could be using this

You must be registered to see the links

No I don't think it's that one. I suspect it's one of the vendors releases for DS, I made a post about a few of them.
https://f95zone.to/threads/loras-for-wildeers-lara-croft-development-thread.173873/post-11676261

 

[felldude]

No I don't think it's that one. I suspect it's one of the vendors releases for DS, I made a post about a few of them.
https://f95zone.to/threads/loras-for-wildeers-lara-croft-development-thread.173873/post-11676261

With all the versions on that site under "creative commons" you never know what you will find hidden in the blender file.
With about 100 outfits and hair styles it damn near crashes my system to load though

 

[me3]

I have loads of questions (for the AI) during all the training, mostly "why" or "WTF"...
Just take these two fairly simple images at a glance nothing really seems wrong, but since i've generally all my test images on the same seed i've started seeing a pattern of what comes from the lora and what things are from the "seed", to put it like that.
Sure the training hasn't worked out because the likeness isn't all that, but as a unique character they'd probably be more than fine. Thing is with what she's wearing, the seed has never produced a jacket/hoodie like that before, so naturally i assumed it must have come from one of the images, specifically the one with a black latex top, belt and guns (this), except when i checked it wasn't in the dataset. Infact only one with clothing was the gym selfie one...I didn't upload them but other images in this run had headsets...i'm pretty certain the AI is just screwing with me at times


This image i just included because with how badly game characters have been treated in games over the last few years, this is probably worryingly close to the next main character in a Tomb Raider game...

 

[felldude]

I have loads of questions (for the AI) during all the training, mostly "why" or "WTF"...
Just take these two fairly simple images at a glance nothing really seems wrong, but since i've generally all my test images on the same seed i've started seeing a pattern of what comes from the lora and what things are from the "seed", to put it like that.
Sure the training hasn't worked out because the likeness isn't all that, but as a unique character they'd probably be more than fine. Thing is with what she's wearing, the seed has never produced a jacket/hoodie like that before, so naturally i assumed it must have come from one of the images, specifically the one with a black latex top, belt and guns (this), except when i checked it wasn't in the dataset. Infact only one with clothing was the gym selfie one...I didn't upload them but other images in this run had headsets...i'm pretty certain the AI is just screwing with me at times
View attachment 2937443 View attachment 2937445

This image i just included because with how badly game characters have been treated in games over the last few years, this is probably worryingly close to the next main character in a Tomb Raider game...
View attachment 2937444

I would recommend turning off GFPGAN when testing a lora, the default strength of .5 can really effect things.

Also you have the terms "(8k, best quality, masterpiece:1.2), ultra-detailed," that can reduce (or rather refine the noise down by a large margin) the number of images in u-net, some terms may completely destroy your lora.

The last lora I trained if I added the term "high color" 10 out of 10 images where not usable.
(I still don't know the reason for sure but I would guess it was the high rank and alpha settings)

For me 1 out of 10 bad images is tolerable, assuming their is 1 out of 10 good images and 8 average.

Spoiler: Example of my recent training failure

You don't have permission to view the spoiler content. Log in or register now.

 

[me3]

So i think i might have to give up on training this into a "real" person. Training it to keep the rendered look doesn't go too badly for likeness, but as soon as you try to get it to be "real" various facial features loose their likeness.
Hair, eyebrows and neck the AI seems to accept, but size of eyes, nose, lips, and general shape of the face it really doesn't want to keep. It always seems to want to "pad" those parts which destroys the likeness.

Anyway, i didn't bother fixing the wrong eye color, the dead expression or the wonky eye, probably not what will draw ppls eyes anyway....

 

[Mr-Fox]

So i think i might have to give up on training this into a "real" person. Training it to keep the rendered look doesn't go too badly for likeness, but as soon as you try to get it to be "real" various facial features loose their likeness.
Hair, eyebrows and neck the AI seems to accept, but size of eyes, nose, lips, and general shape of the face it really doesn't want to keep. It always seems to want to "pad" those parts which destroys the likeness.

Anyway, i didn't bother fixing the wrong eye color, the dead expression or the wonky eye, probably not what will draw ppls eyes anyway....
View attachment 2938122

boobaa!

 

[felldude]

So i think i might have to give up on training this into a "real" person. Training it to keep the rendered look doesn't go too badly for likeness, but as soon as you try to get it to be "real" various facial features loose their likeness.
Hair, eyebrows and neck the AI seems to accept, but size of eyes, nose, lips, and general shape of the face it really doesn't want to keep. It always seems to want to "pad" those parts which destroys the likeness.

Anyway, i didn't bother fixing the wrong eye color, the dead expression or the wonky eye, probably not what will draw ppls eyes anyway....
View attachment 2938122

Have you tried using your newly trained lora at .3-.6 on your source training image with karras sampler at .2-.5 (Img to Img)

It might have the mix your looking for.

Spoiler: Example from WIP Lora

You don't have permission to view the spoiler content. Log in or register now.

 

[felldude]

Using the method above I tried just the face (It's a 10x Upscale so asking a lot)
I don't think either lora learned the facial geometry but at that distance with 3d it would be difficult to learn in my opinion

I didn't test across all version's in this case it was the 1.0 and beta

You must be registered to see the links

Lara Lora Beta



In a source image render it appears the Wildeer 1.0 learned the lipstick and facial expression of the source better.

Wildeer

Lara

 

[me3]

"replicating" the training images doesn't work too badly i guess, but trying to force different "elements" on those prompts is a bit of a challenge.
Which epoch would you pick as the "closest match"?
While it's probably not needed i've included the exact training image for reference, but i think it should be possible to recognize it regardless.




Unfortunately the forum rescaled the width by about 1300px so things got a bit blurry

Spoiler: training image

You don't have permission to view the spoiler content. Log in or register now.

This grid is of a slightly altered prompt of the one above. added "full body" with weighting and "laying" alone with the wide format to fit, there's some obvious issues, only negative prompt is to get right of the "wet look", so much can probably corrected there. Pretty certain some of those "poses" require some hospital visits before and definitely afterwards.

 

[me3]

i think i just need to start using the sample images instead of any loras cause when they turn out like that it usually isn't reflected in the lora :/
...maybe i could add it to the training data....it's less blurry/noise than some of the images

 

[Mr-Fox]

i think i just need to start using the sample images instead of any loras cause when they turn out like that it usually isn't reflected in the lora :/ View attachment 2942696
...maybe i could add it to the training data....it's less blurry/noise than some of the images

This is exactly what I have been doing for a little while, instead of bothering with training a Lora. I use the image as input in controlnet. I typically use line art realistic and then T2i adapter with color grid.
This is how I created that classic Lara Croft with Angelina's face in roop. There are other interesting controlnet models such as revision or Ip adapter.

You must be registered to see the links

Also don't overlook using photoshop. I use it to prepare the source image. Both resizing and cropping as well as doing some editing. Then I use it in controlnet to generate my image, then I use photoshop to fix minor details and adjusting color, lighting, possibly improve clarity and what not, sometimes add a little vignette etc.

 

[me3]

There are some obvious issues with these, but at least some of it might be fixable with prompting/weight if anyone is interested.
Seems to work very well for recreating training images
The redish hair is just me testing how it responds

You must be registered to see the links

As a note, text encoder rate is 10x what was intended so probably need to set that like 60% or lower, it's the sort of fun thing you get when you're just copy pasting values and hitting numbers without looking...

 

[Mr-Fox]

There are some obvious issues with these, but at least some of it might be fixable with prompting/weight if anyone is interested.
Seems to work very well for recreating training images
The redish hair is just me testing how it responds
View attachment 2953228

View attachment 2953218 View attachment 2953219 View attachment 2953220

You must be registered to see the links

As a note, text encoder rate is 10x what was intended so probably need to set that like 60% or lower, it's the sort of fun thing you get when you're just copy pasting values and hitting numbers without looking...

Awesome work.

 

[me3]

Awesome work.

well...not really, it might be in some limited way be usable with large tweaks to the weights, mainly text, but that's never a good sign.
I initially thought was because of my wrong lr setting, but i've set that and redid the whole ~18k step training...sigh...and the text encoder is still going nuts for some reason.
As a note, the gui for kohya_ss has a "stop text encoder" setting, well, it's pointless, if you use it you just get told it's not added yet.

 

[Mr-Fox]

well...not really, it might be in some limited way be usable with large tweaks to the weights, mainly text, but that's never a good sign.
I initially thought was because of my wrong lr setting, but i've set that and redid the whole ~18k step training...sigh...and the text encoder is still going nuts for some reason.
As a note, the gui for kohya_ss has a "stop text encoder" setting, well, it's pointless, if you use it you just get told it's not added yet.

I was only complimenting your effort my friend. The first example image is also true to the source material and good looking. Improvements can always be made, and with each attempt we learn from the failures and gets a little closer to the endgoal.
Imo we learn more from failures than successes at least in general terms.

 

[Mr-Fox]

You can probably get a good result with your Loras if you use one of the source images in controlnet lineart realistic.
There are other controlnet models that could be used as well. Using a workflow with controlnet doesn't exclude a need or the benefit of Loras. One doesn't exclude the other. I often use Loras in combo with controlnet.

 

[ToxicLemming]

I'm with you 100% on the use of a lora, if it can not generate new images and new scenes then it holds little use for me.

Your on your own about the AI rebelling, if floating point numbers start being anything more then just math...then my meds are finally working...or not working.

You mention learning, I read the entire

You must be registered to see the links

Mr-Fox posted, I still have it pulled up. My initial reaction was disappointment because I was hoping to learn something new about XL and did not (Also why I edited my first response back).

I also noticed so outdated/misinformation about the adaptive optimizer and I still haven't found the paper buy the folks that worked on it but it compares constant and cosine even with adaptive.

That doesn't make the article less informative, its well written.
The same can be said for the

You must be registered to see the links

I linked too from the maker of the tool I used and you likely did also to train.
Sadly he has a fraction of the views of some other videos on YouTube.

Both myself and Mr.Fox offered to different approaches for cleaning up images, he said he wasn't going to name names, and he probably didn't even have me in mind when he mentioned it, but I have done what he said and over correct (bake) a Lora using image to image to enhance a data set.

I could reading/misinterpreting your response wrong, it seems like you did not find any of this helpful, but perhaps someone else will.

I combined the training I did with the data set 3 here and a lora I was working on for some interesting results.


View attachment 2928245

Unfortunately it seems to have dislocated her hips

View attachment 2928244

Huh. I've never seen a Vaginass before.....