I Thought Better Models Would Fix AI Image Editing. I Was Wrong.

For a while, I assumed the biggest problem in AI image editing was obvious: the model just wasn’t good enough yet. If the output changed a face too much, use a better model. If the product logo became distorted, use a better model. If the lighting looked strange, wait for the next model release. That explanation is comforting because it turns every failure into a model-quality problem. But while working on an AI photo editing workflow, I kept running into a different conclusion: the harder problem is expressing what is allowed to change — and what is not. That sounds like a UX detail. I now think it is the core abstraction of AI image editing. Generation and editing are not the same task Text-to-image generation is wonderfully forgiving. If I ask for: A cinematic photo of a red sneaker on a concrete pedestal, soft studio light, dark gray background. there are thousands of outputs that could reasonably satisfy me. Editing is different. Imagine I already have a product photo. I want to keep the exact shoe, its proportions, material, logo, color, and camera angle — but replace the background with a cleaner studio scene. Now the request is not simply: Generate a better product photo. It is closer to: Change the environment, but preserve the product identity. That second half is where a surprising amount of the difficulty lives. You can see the difference clearly in a before/after product edit: The user is not asking the system to be creative everywhere. They are granting creativity inside a boundary. That led me to a simple mental model: // Conceptual model — not an API contract const editRequest = { source: originalImage, change: "Replace the background with a clean studio setup", preserve: [ "product shape", "logo", "material", "color", "camera angle" ] } Enter fullscreen mode Exit fullscreen mode The more I thought about it, the more this looked less like prompting and more like constraint management. “Change this” is only half of the prompt Most people naturally describe the desired change first: remove the people in the background make the room Scandinavian turn daytime into night change the jacket to black add warm cinematic lighting But for editing, the preservation requirements are often equally important: keep the face unchanged keep the original pose preserve the product logo do not move the furniture keep the camera perspective keep the typography intact This is the pattern I now encourage in the interface and in prompts: Change X. Preserve Y. It is almost embarrassingly simple, but it produces a much clearer contract between the user and the model. A room redesign is a good example. “Make this room more modern” is underspecified. Does the user want to replace the sofa? Move the windows? Change the floor plan? A better request is something like: Redesign the room in a warm minimalist style. Keep the room layout, windows, camera angle, and major furniture positions unchanged. That is much closer to how a human art director would brief an editor. Reference images are really another kind of constraint Another thing I initially thought about too narrowly was reference images. It is easy to treat them as “more prompt.” In practice, they are more useful as a visual contract. A reference can answer questions that are annoying to describe in text: What shade of blue? What kind of lighting? How dense should the texture be? What does “minimalist” mean in this specific project? Which composition should the edit move toward? In the workflow I’m building with ClipLumi, users can add up to three reference images alongside the main editing instruction. The important part is not the number three. The important part is that the UI acknowledges something developers already know from multimodal systems: language is not always the best representation of intent. Sometimes the most precise instruction is another image. Natural language does not eliminate the need for controls There is a tempting product idea in generative AI: If the model is smart enough, every setting should disappear. I don’t think that is always true. Some choices are not semantic. They are production constraints. A user may know exactly what they want: 16:9 for a website hero 9:16 for a mobile story four variants instead of one a faster draft before spending time on quality a specific model because it behaves better for a certain task Those are not failures of natural language. They are legitimate parameters. So the current ClipLumi editor keeps explicit controls for things such as model selection, aspect ratio, output count, resolution, quality, fast mode, and style. The design question becomes: Which choices belong in prose, and which choices deserve a control? My current rule of thumb is: Use language for intent. Use references for visual direction. Use controls for production constraints. That separation has been more useful to me than trying to turn everything into one giant prompt box. The goal was not “Photoshop with AI buttons” This also changed how I thought about the product itself. I did not want to rebuild a traditional editor in the browser and then add an AI button to the toolbar. Traditional image editors are powerful because they expose the mechanics directly: selections, masks, layers, blend modes, brushes, curves, transforms. For many everyday edits, though, the user does not care about the mechanics. They care about the outcome: Remove those people, but keep the subject and the scene intact. The whole point of a prompt-driven editor is that the user should not need to manually paint a mask just to communicate that intention. That is why the product direction became less about “AI features inside an editor” and more about editing through constraints, references, and natural language. Better models still matter — just not in the way I expected None of this means model quality is irrelevant. It matters a lot. There are still cases where generative editing is frustrating: small text and logos can deform hands and fine geometry can change unexpectedly multiple edits in one request can interfere with one another preserving identity across aggressive transformations is difficult the same prompt can behave differently across models A stronger model can reduce those failures. But a better model cannot fully rescue a badly specified task. If the interface gives the model permission to reinterpret the entire image when the user only wanted one small change, the product has already made the job harder than necessary. This is why I increasingly think the next improvement in AI creative tools will not come only from larger models. Some of it will come from better ways to represent user intent. The bigger question So here is the question I’m curious about from other developers building with generative models: Should future AI editors expose more model controls, or should they hide more of the model behind higher-level constraints? In other words, are we moving toward: increasingly sophisticated prompt boxes, increasingly sophisticated traditional editors with AI features, or a new kind of interface where users mainly define what may change, what must remain stable, and what references the system should follow? I’m currently betting on the third direction. That is the idea I’m exploring with ClipLumi: upload an image, describe the change, state what must remain intact, optionally add references, and keep enough explicit controls to make the output usable in a real workflow. It is still imperfect, and that is exactly why I find the problem interesting. If you were designing an AI image editor today, what would you expose to the user — and what would you hide? I’d genuinely like to hear how other people here are thinking about this boundary.