I filed an issue: https://github.com/adamkdean/gpt-unicorn/issues/2

"Running this project daily doesn't make sense if GPT-4 is not being constantly updated"

With a suggestion to run it monthly instead, and generate 16 images at a time, and backfill it for GPT3 and GPT3.5.

Did you generate a bunch all at once before starting to get some idea of what the natural variance looks like? I would think it's important to verify some level of progression over time, because with the current four it seems entirely possible that the examples could have all been generated at the same time with no changes to the model.

Sampling a single noisy sample from a model that doesn’t update that often is hardly correlated with the claim of “Daily exploration”.

The unicorn example is discussed at length in Bubeck's recent talk:

https://www.youtube.com/watch?v=qbIk7-JPB2c#t=22m6s

As is noted in the paper from which this is inspired from: GPT-4's image generation capabilities were severely diminished by the instruction/safety-tuning process. Unfortunately this means the currently available model from the API won't be very capable - certainly not as capable as the early version of GPT-4 that Microsoft had access to.

edit: I'm specifically referring to the "image generation by trickery (e.g. SVG)" technique being diminished. Other tasks were diminished as well though - is my understanding.

Why would the model change over time when asking the same question? Just it's generation dataset for generating similar images? Or is this just tracking GPT's explicit model improvements over time?

Look at this majestic beast GPT-4 just drew! https://imgur.com/a/ogZ6qI0

Interestingly, I think you often get better results with ChatGPT web interface than the API (the OP used a system message), but for a funny reason.

Because ChatGPT typically first responds by insisting it can't actually draw, it ends up accidentally chain-of-thought prompting itself into drawing better. "I can't draw, but I can describe a detailed drawing for you in words." So instead of winging it with the SVG code and getting confused, it goes into it with a plan.

This is how it described the unicorn I linked:

__User__ Draw a unicorn!

__ChatGPT__ As an AI language model, I am unable to create visual images directly. However, I can describe a detailed image of a unicorn for you in words.

Picture a magnificent unicorn, standing tall and proud in a serene setting. The unicorn's coat is a shimmering, pure white, and it glistens under the warm sunlight. Its body is strong yet elegant, with a graceful arched neck and powerful, slender legs. The hooves are cloven and emit a faint, otherworldly glow.

The most striking feature of the unicorn is its majestic spiral horn, which emerges from the center of its forehead. The horn is long, slender, and gently tapers to a sharp point. It is iridescent, with colors swirling together like a mesmerizing opal. The unicorn's mane and tail flow behind it like silken strands, shimmering with an ethereal pastel rainbow of colors.

Its eyes are large, expressive, and full of wisdom, with a depth that speaks of untold secrets and ancient magic. The unicorn's ears are pointed and alert, picking up the slightest rustle of leaves or the gentlest of whispers. The unicorn's expression is one of serenity, exuding an air of peace and tranquility that surrounds it like a gentle embrace.

Though it is a creature of great power, the unicorn carries itself with a sense of humility and grace that inspires awe and respect in all who behold it.

__User__

What's the SVG for the image you just described?

__User__

continue (to finish the file)

Some more examples here, but with many iterations of "Improve this" over and over:

https://twitter.com/jonathanfly/status/1639657098605740035

      \\
       \\
        \\  ,__,
         \\ (oo)____
            (__)    )\\
                 ||--|| *

This is a great rorschach test. Show these four images to someone hyping AI, and if they see evidence of a growing/emerging intelligence, you can diagnose them as being wholly unqualified to comment on anything related to AI.

I asked gpt4 to "draw a unicorn using svg", and it gave me way better results:

<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 100 100" width="100" height="100"> <circle cx="50" cy="45" r="15" fill="#FFC0CB" /> <circle cx="50" cy="65" r="18" fill="#FFC0CB" /> <rect x="44" y="33" width="12" height="15" fill="#FFC0CB" /> <rect x="40" y="15" width="3" height="20" fill="#FFD700" /> <circle cx="45" cy="40" r="3" fill="black" /> <circle cx="55" cy="40" r="3" fill="black" /> <path d="M35 70 Q50 55 65 70 T95 70" stroke="black" stroke-width="2" fill="none" /> </svg>

> The idea behind GPT Unicorn is quite simple: every day, GPT-4 will be asked to draw a unicorn in SVG format. This daily interaction with the model will allow us to observe changes in the model over time, as reflected in the output. Is it useful to do this every day? Correct me if I'm wrong, but my understanding is that OpenAI does not update the models available in production incrementally on a day-to-day basis.

FYI, I forgot to make the data persistent[1] so the first five images were sadly lost just now as I made a few updates to the repo. Apologies for this.

[1] https://github.com/adamkdean/gpt-unicorn/commit/08e400dbc437...

I had a short conversation with GPT-4 about a unicorn. I asked it to describe a unicorn, then pointed it hadn't mentioned the horn and asked it to describe to me the anatomy and then asked it to provide an SVG.

I got this: https://www.svgviewer.dev/s/yldwue0Q

I would argue this is better than what they're achieving.

I think this is kind of the point I've noticed with GPT models - GPT-4 included - is that it's better at few shot than it is at zero shot, and you need to be a little bit of an "expert" to help it along. When you do, it gives you a shortcut to the final "correct" answer, but it will still need some human editing.

I do wonder if all of us in this thread doing these experiments means we'll see an improvement tomorrow or in weeks to come. I'm also keen to see what GPT-5 looks like in this regard.

EDIT: I noticed some people asked to "make it better" and it did. I tried, but got nearly identical output, so I moved along a bit:

Me: That's too similar. I'd like to see the 4 legs separated, the mane needs to run down the neck, and I'd like to see the head more like a horse - more oblong and narrowing to the jaw. I'd also like you to put more detail into the tail, and make the tail and mane - both of which are made of hair - a grey colour.

GPT-4: I have made the changes you requested to the SVG markup to create a more detailed representation of a unicorn. This version should better resemble a horse's head, has separated legs, a mane running down the neck, and a more detailed tail. The mane and tail are also colored grey, as requested.

https://www.svgviewer.dev/s/WQcdOhVP

That output is... better? Maybe? I mean, it's followed my instructions (apart from the legs), I guess...

if GPT-4 will update based on recent web training data, the fact that people are bringing much more attention to the "draw a unicorn" task magnifies the chance someone will have posted a perfect version of an svg unicorn, leading the model to leverage that rather than the aim of this experiment which I imagine is GPT-4's capacity to extrapolate.

EDIT: Also it makes no sense to constantly retry it every day on the gpt-4-0314 model, since OpenAI specified that that is a snapshot model that will not be updated.

Would be great if these days would have dates as well. Otherwise, there's little use of "Day 69". If I could see "Day 69 (June 21 2023)"

Thought it might do better if asked it to use the javascript canvas api. The comments it added seemed promising. But the end result wasn't much better.

Prompt was: Draw a unicorn using the javascript canvas api

https://jsfiddle.net/za7Ltk65/ I added the minimal css and html, but the JS is verbatim what it spit out.

Hmm, tempting to make a thing that just loops:

description = gpt4("describe <yesterday's output>")

and chain that into

new_result = gpt4("<yesterday's output> was described by a human as <description> but it should have been visibly apparent as a unicorn. provide SVG output that <extra blah blah>")

and post that as today's output. I wonder what we'll get. I'll do it manually.

Are they banking on OpenAI updating their model every day, or just prompting the same thing everyday wishing for a different outcome?

Would be useful if the prompts used to generate the drawing code were included in the site.

I got it to generate a cube and an apple in .obj format, legit enough to import into maya, though the apple looked more like a gem stone

If this becomes popular, it will become a target for the team working on GPT-5.

If you ask it to write GLSL code implementing Signed Distance Fields, it can generate a 3D rendering of a unicorn as well. Just paste it into ShaderToy. I asked it to implement animation as well, for the legs, and it manage to do something although incorrect.

I'm surprised nobody made a GPT to generate startup ideas yet.

How can a model trained on text know what a unicorn looks like

Might as well make a Twitter account! or get AutoGPT to do it.

It can draw a penis fyi

Perhaps you should ask it to draw you a sheep.

Is anyone else also getting tired of seeing "GPT" prefix / suffix in the name of 90% new AI-related products?