top of page
Search

Flux Lora Training Tutorial with RunPod's FluxGym Template

  • locallab
  • Sep 27, 2024
  • 5 min read

The world of AI image generation is buzzing with excitement, especially with the emergence of powerful Flux models like Schnell and Dev. These models are turning heads with their ability to produce incredibly high-quality images, opening doors to a new era of artistic expression. But what if you could take these models and personalize them, tailoring their creative output to match your unique vision? That's where LoRAs come in, and today, we're going to unlock their incredible potential.


In this guide, I'll walk you through the process of easily training your own custom LoRAs on these cutting-edge Flux models using a brand new tool: the FluxGym RunPod template. It's a game-changer, simplifying LoRA training while making it incredibly cost-effective.


Why LoRAs? Why FluxGym?


LoRAs, or Low-Rank Adaptations, are like specialized training modules for AI models. They let you fine-tune a model's output, teaching it to generate images in a specific style or focus on particular concepts. Imagine training a LoRA on images of a particular artist's work, giving you the power to generate images reflecting that artist's signature style!


FluxGym, built upon the innovative Cocktail Peanut FluxGym repo, streamlines the entire LoRA training process. It leverages the power of the RunPod cloud platform, providing a user-friendly interface and access to robust GPUs. This combination makes custom LoRA training accessible, even if you're not a tech wizard or have a limited budget. We're talking about potentially spending less than 50 cents for a complete training run, depending on your chosen GPU!


If you want to watch the video tutorial instead, watch here:




Setting Up Your Artistic Playground on RunPod


Let's begin our LoRA training journey. First, you'll need a RunPod account. If you don't have one already, head over to their website and quickly create one. It's a breeze!


Next, we'll use the FluxGym RunPod template – you can find the link in the resources section below. This template takes you directly to the Deploy GPU Pod page on RunPod, where you'll be presented with a smorgasbord of GPU options to choose from. Each GPU comes with its own hourly cost, so pick one that fits your budget.


For this walkthrough, we'll be using the RTX A4500. It's a great balance of power and affordability, boasting 20GB of VRAM and 31GB of RAM – more than enough for our LoRA training. Plus, the A4500 usually has good availability.


Double-check to ensure you've selected the right template: "Local Lab FluxGym LoRA Training with Kohya ss". If you want to personalize your setup – for example, increase the storage space – click the "Edit Template" button and adjust the container disk and volume disk storage sizes as needed. Just remember: container storage is temporary and data stored there will be lost when your pod terminates, while volume storage is persistent and remains even after your pod shuts down.


Once you're happy with your configuration, click the "Deploy" button. RunPod will take over, building your pod and automatically running the scripts to install dependencies and download the required models. This process may take 10-15 minutes due to the size of the models, but you can track the progress in the logs tab.


Launching Your Training Studio: The FluxGym Web UI


When the installation is complete, click the "Connect" button and then the "HTTP Service 7860" button to launch the FluxGym Web UI directly in your web browser. You should see a sleek, user-friendly interface ready for action.


Now it's time to configure your LoRA training settings. First, give your LoRA a memorable name. Next, enter the trigger word or sentence that will activate your LoRA when generating images. This is how the AI model will learn to associate your training images with a specific style or concept, so choose a trigger that's clear and relevant.


For this demonstration, we'll stick with the default values for the other training settings. However, feel free to explore and tweak them based on your unique LoRA training goals and desired outcomes. For example, experiment with the number of "Repeat trains per image" or "Max Train Epochs". If you click on the "Advanced settings" tab, you'll uncover even more parameters to fine-tune your training.


Feeding Your AI Artist: Adding Training Images


Next, add your carefully curated training images. Simply drag and drop them into the "Dataset" section. Aim for around 15-20 high-quality images for optimal results.

Once uploaded, FluxGym will automatically caption your images using the impressive Microsoft Florence 2 model. This built-in feature takes the hassle out of manual captioning, saving you time and effort.


Unleashing the Creative Process: Start Training


With everything in place, click the "Start Training" button. The magic begins! You can monitor the progress and view the training logs in the designated sections at the bottom of the page.

The training process may take a bit of time, but it's generally much faster than training on platforms like Google Colab. Expect a wait time of around an hour or so, depending on your chosen settings, dataset size, and the VRAM you're using.


Sharing Your Masterpiece: Downloading and Using Your LoRA


Congratulations, your LoRA is trained! Now, let's download it to your device and put it to use. The easiest way to do this is to first push your LoRA to your Hugging Face account.


Navigate to the "Publish" tab, log into your Hugging Face account (or create one if you haven't already), and create a model page for your LoRA. Next, obtain an access token. Click on your profile picture, go to settings, and then click on "Access Tokens" in the left menu. Create a new token and ensure you give it "Write" access, as this is essential for pushing your LoRA model.


Copy the access token, head back to the FluxGym Web UI, paste it into the "Hugging Face Token" field, and click "Login". In the "Trained LoRAs" section, select your freshly trained LoRA from the dropdown menu. Type the name of your Hugging Face repository into the "Repository Name" field, and click "Upload to Hugging Face".


Once the upload is complete, your LoRA will be available on your Hugging Face model card. You can download it from there, then place the downloaded LoRA file into your LoRA folder within your preferred image generation program.


Experiment, Create, and Share!


And there you have it! You've successfully trained a custom LoRA on a powerful Flux model, ready to personalize your AI image creations. Experiment with different training settings and datasets to refine your LoRAs and unleash your artistic vision. Don't hesitate to share your experiences and results with the community. Welcome to the exciting world of AI artistry!


Resources:



Happy creating!

Comments

bottom of page