Generative AI has opened up incredible new creative possibilities by enabling anyone to describe a visual scene just with words and have it brought to life automatically. However, even with today‘s most advanced systems like Midjourney, the initial auto-generated images rarely match our precise expectations.
That‘s where inpainting comes to the rescue. This new Midjourney capability lets you directly edit and modify parts of an existing AI-generated image by "painting over" certain areas with new text prompts.
In this in-depth guide, we‘ll cover everything you need to know to harness the power of Midjourney inpainting for your own AI art and content creation.
An Introduction to Inpainting
Inpainting is like having a magic touch-up brush for AI art. Instead of needing advanced artistic skills or photo editing software to fix imperfections or change elements in a picture, you simply select the area you want to alter and provide a new text description.
Some key uses and benefits of inpainting include:
- Placing imported objects into scenes – Cut an object from one image and paste it into another context.
- Changing seasons or time of day – Easily modify the environment and lighting in a landscape.
- Altering small details – Fix artifacts, adjust colors, remove unwanted elements.
- Creating surreal combinations – Blend disparate images and concepts into one.
The technology works by training AI systems like Midjourney on vast datasets of image regions and their surroundings. It learns to generate new pixels that smoothly continue existing patterns and textures while matching the content described by your inpainting text prompt.
However, like all AI abilities today, it has limitations…
Some types of edits will likely come out looking quite rough around the edges. But as neural networks continue to advance, the quality and flexibility of inpainting will only get better.
Now let‘s dive in and see it in action!
Getting Started with Midjourney Inpainting
Using inpainting in Midjourney requires just a few quick steps to set up first.
-
Join the Midjourney Discord server if you haven‘t already. This is where you‘ll generate images and access the inpainting interface.
-
Go to
/settingsand enable "Remix mode". This allows you to further edit and build on existing generations. -
Generate a base image with any prompt as usual. Once it‘s created, click the
Upscalebutton one or more times to enlarge and refine it. -
Hover over the upscaled image and click the
Varybutton. Then selectVary (Region)from the popup menu. -
This opens the inpainting editor! Now it‘s just a matter of choosing an area and providing a new textual description to modify it.
The interface includes both rectangular and lasso selection tools to highlight regions of the image you want to inpaint over. Feel free to select multiple distinct areas to edit.
Some key tips when selecting regions:
- Avoid cropping out narrow details on object edges
- Leave some overlap/context rather than tightly cropping
- Don‘t worry about minor spillover – the AI won‘t alter unrelated areas
Once you have a region selected, simply type a text prompt in the input field and hit Enter to have the AI generate a new inpainted area blending seamlessly with its surroundings.
You‘ll get four variant results to pick from, just as usual with Midjourney images. Choose your favorite, or try re-prompting further until you get the desired modification.
Now that you know the basic workflow, let‘s look at some real examples of inpainting in action!
Midjourney Inpainting Use Cases and Results
While inpainting is still an early capability, it can produce impressively realistic edits when used skillfully. To get a feel for what it can do, I tried a variety of prompts and modifications spanning backgrounds, objects, textures, lighting, and more.
Here are some highlights along with the steps I took and observations from each test.
Use Case 1: Changing Seasons in a Landscape
Base prompt: A colorful tree in fall with red and yellow leaves next to a lake and mountain range behind, matte painting trending on ArtStation
This generated a beautiful fall mountain landscape as a starting point. But I wanted to see if I could transition it to a summer scene instead…
Inpaint prompt: lush green tree full of green leaves
Area selected: Tree and ground below it
The inpainted summer tree came out quite well! The leaves and shadow all matched nicely. It even realistically varied some grass colors around the tree base.
There are artifacts around the edges if you look closely, but overall it did an impressive job considering how different the new element is from the original autumn colors.
Use Case 2: Alter Lighting and Textures
Base prompt: Hyperrealistic intimate cozy living room interior with armchairs, decorations, bookshelves and warm lighting
Nice mood, but I wanted to give this room a cooler nighttime vibe…
Inpaint prompt: Same living room with all blue lighting and textures
Area selected: Entire room
Remarkably, inpainting the whole room with a pretty vague "make everything blue" prompt worked decently! The wall/floor textures changed nicely. And while not everything took on a blue tint, the lighting shifted convincingly to feel like moonlight.
This shows the power of inpainting for environment adjustments. And selecting larger regions seems to work better than small details.
Use Case 3: Retouching Portraits
Base prompt: A majestic portrait painting of a female astronaut in spacewalking gear floating gracefully above Earth
I really liked this one overall. But let‘s see if we can tweak the space suit to be more NASA-accurate…
Inpaint prompt: Iconic white and orange NASA astronaut suit
Area selected: Entire body
Unfortunately this attempt came out quite rough and distorted. It seems inpainting struggles more with fine textures and intricate shapes like faces and outfits. The transparency around the edges makes it obvious where my selection box was drawn.
While imperfect, I‘m still impressed it handled as well as it did trying to adapt the style of the suit to this painting aesthetic.
Use Case 4: Blending Disparate Objects
Base prompt 1: An astronaut smiling at the viewer in front of nebula and stars
Base prompt 2: A large green alien spaceship getting ready to abduct something
Let‘s transport our friendly astronaut onto the mysterious alien ship!
Inpaint prompt: Smiling astronaut floating inside alien spaceship bridge
Area selected: Spaceship window
This was one of my favorite inpainting tests. The added astronaut fits nicely into the exposed bridge. And while the blending isn‘t seamless, it works passably well considering I essentially cut and pasted from two totally unrelated renderings.
It‘s a great example of the creative possibilities once you view AI generation as an artistic collage medium rather than expecting photorealism. Don‘t be afraid to experiment!
Tips for Improving Your Midjourney Inpainting Results
As you can see from these examples, Midjourney inpainting is impressive but imperfect. Based on my testing, here are some tips to help your edits come out cleaner:
- Focus on environments rather than precise objects – It handles background textures and lighting changes better right now.
- Use larger region selections when possible – Tight crops on intricate shapes are more likely to get artifacts.
- Match image styles for inserted elements – If your base image is realistic, describe additions realistically rather than as paintings or illustrations.
- Try multiple prompt phrasings – Small tweaks can make a big difference. Emphasize how the new content interacts and blends with the surroundings.
- Leave some of the original context – Completely covering up old elements risks odd-looking edges.
And as always when prompting AI art, use descriptive and concrete language. Have patience trying different inputs, and let the technology meet you halfway rather than expecting human perfection!
What Does the Future Hold for AI Inpainting?
While Midjourney‘s current inpainting capabilities may seem simplistic compared to advanced tools like Photoshop or manual illustration, this is just the beginning.
Neural art generation is improving faster than ever across realistic image editing, creative visual blending, working with 3D spaces, and even first steps into video.
In the months and years ahead, expect inpainting quality and flexibility to grow dramatically. Other AI services will continue catching up to and competing with Midjourney as well.
Some likely advances that will hugely expand creative possibilities:
- Smoother blending – More seamless edges and natural texture adaptation
- Object insertion and removal – Moving individual elements around rather than whole regions
- Integration with upscaling/downscaling – Better results editing at different zoom levels
- 3D inpainting – Modifying full 3D rendered scenes, not just flat images
- Video inpainting – Keyframed movie editing by describing changes over time
Along with better technology, the user interfaces will improve towards more intuitive and detailed control over adjustments. Think something like "grab and drag" tools to easily pick elements out of an image or place new ones. That could almost turn generative AI into an automated photoshopping assistant!
Ultimately, expect these advancements to keep opening new creative doors while lowering the barrier to stunning multimedia editing and production. But with new power comes responsibility…
The Ethical Implications of Powerful Inpainting
As AI editing abilities grow exponentially stronger in the coming years, manipulating images and video will become exponentially easier.
We‘re not too far from a world where even a quick cell phone doodle combined with text descriptions could produce a photorealistic scene mixing concepts in ways that fundamentally deceive human perception.
The dangers of Malicious actors generating fake yet convincing:
- News reports and photos spreading misinformation
- Compromising images harming public figures
- Impersonating identities for dangerous fraud
are obvious and rightly concerning. However, the incredible creative boons of democratizing experience creation cannot be ignored either.
As with any powerful technology, the responsibility lies with human creators and consumers to navigate these AI waters wisely and ethically. Support efforts from companies like Midjourney to enhance attribution and verification tools. Think before you generate, consider carefully before sharing anything misleading, and educate others.
Our society is far from defenseless against this challenge. We‘ve adapted policy and social norms to handle new media revolutions like photography and video editing in the past after all! This time need not be any different if we face it clear-eyed rather than reactionary.
Now go grab this exciting new AI painting assistant for yourself with some Midjourney inpainting exploration! Just use wisdom, kindness and clarity of purpose along the way.
