Great article minimaxir. Couple points I should make.
From the article:
> the infamous Make me into Studio Ghibli prompt
I've done a fair amount of style transfer testing with NB and NB Pro.
We’ll probably never know whether Google has added deliberate roadblocks or if it’s simply not well represented in the training data, but if you’re at all familiar with Ghibli’s work (Nausicaä, Mononoke, Porco Rosso), the resulting profile image scans more like a relatively vaguely anime-esque illustration than anything even remotely resembling Miyazaki’s style. You’d never be able to pick this image out of a lineup and say it’s a Ghibli drawing.
Even trying to prompt with clear stylistic references doesn’t help. For example, here’s some early promotional artwork from the NES Legend of Zelda: it’s highly stylized and very distinctive. The resultant images (while nice and clean looking) bear none of the hallmarks of the originals.
As far as grid generation goes, I've actually experimented with this for creating sprite sheets. I found that in many cases, NB Pro performed even better if you provided it with a blank 1024×1024 image that already had evenly spaced gridlines, and then used it in an image-to-image mode to fill in the squares with the respective sprites.
vunderba•1mo ago
From the article:
> the infamous Make me into Studio Ghibli prompt
I've done a fair amount of style transfer testing with NB and NB Pro.
We’ll probably never know whether Google has added deliberate roadblocks or if it’s simply not well represented in the training data, but if you’re at all familiar with Ghibli’s work (Nausicaä, Mononoke, Porco Rosso), the resulting profile image scans more like a relatively vaguely anime-esque illustration than anything even remotely resembling Miyazaki’s style. You’d never be able to pick this image out of a lineup and say it’s a Ghibli drawing.
Even trying to prompt with clear stylistic references doesn’t help. For example, here’s some early promotional artwork from the NES Legend of Zelda: it’s highly stylized and very distinctive. The resultant images (while nice and clean looking) bear none of the hallmarks of the originals.
https://imgur.com/a/failed-style-transfer-nb-pro-o3htsKn
As far as grid generation goes, I've actually experimented with this for creating sprite sheets. I found that in many cases, NB Pro performed even better if you provided it with a blank 1024×1024 image that already had evenly spaced gridlines, and then used it in an image-to-image mode to fill in the squares with the respective sprites.
https://imgpb.com/rJqARJw