FrameIt - AI Art Frame

In a first year business class, I was part of a group that pitched a digital poster, built on an E-Ink panel. I loved the idea of hanging some art without committing to a single piece for all time. Every few years I'd check if the panels had seen a price drop that made the venture seem worthwhile, but they remained sky high. But, now that AI-generated images have proliferated, I thought I'd revisit the idea of a digital art installation. So the aim is to get some AI powered content up on the wall with minimal effort and minimal cost - hacky, duct-tape-style methods are welcome. Ideally, I’d like to simply describe an idea for a piece and have it seamlessly appear on the wall.

The Build

E-Ink was initially appealing for its characteristics which mimic the look of actual paper or canvas. While E-Ink remains expensive, Samsung now offers "Frame TVs" that seem to fit the bill. These TVs are designed to serve as digital wall hangings when they're not in active use, with a matte finish, slim wall mount, and an optional frame that magnetically snaps around the screen. Samsung even has their own "Art Mode" built-in, although you'll need to subscribe to their service for $5/month or manually add and remove images. In the interest of cost I didn’t go down that route here - the AI generated images provide limitless options for artwork anyways. I ended up going for the smallest such TV available at 32", which offers 1080p resolution.

For the AI art generation, I use fal.ai as my tool of choice. I had some previous experience toying with it, so I already had some credits on the service. And, they believe their Stable Diffusion XL generation is the "fastest on the planet". But, if you'd rather use Replicate or some other provider, undoubtedly that would be fine, too. To manage costs, I generate at a relatively low resolution (1024x576) and upscale afterwards (using ESRGAN). If I directly generated at 1080p, my test runs showed each generation would cost around $0.07, while the 2-step process costs around $0.006 for a ~10x savings. The resulting 2-step script is pretty brief, simply calling a couple REST RPCs. Here's an excerpt:

Next, I considered several options for calling the required APIs and pushing the image to the TV. I couldn't find an easy way to push images programmatically to the TV's built in art mode, taking that option off the table. Briefly I thought I might want to throw together an app for my iPhone, but I really didn't want to dive into iOS development if I didn't need to. I had a Pi Zero W lying around unused, so I decided to use that. It's straightforward to set up, although it means the TV isn't officially in "Art Mode'' while it's connected. Directly adjusting the settings for low brightness creates a similar effect. The utility fbi is used to throw the generated images on screen.

Finally, with the Pi acting as the workhorse, I just added a Shortcut to my iPhone to run the necessary commands over ssh. I was delighted to find that an ssh Shortcut was built right in! So we can just call a couple commands to make the whole thing work. This way, we can easily type or use Siri to kick off a new "framing". While building this shortcut, I did run into a few issues and irritations. First, Siri would hang around and complain that the whole thing failed, even when it succeeded, but explicitly dismissing Siri after the required inputs are collected resolved that issue. Then, the "one-time setup" inputs simply didn't work if I selected the "Text" type. I used a dictionary to work around that. But overall, it didn't take long to get up and running, and so it perfectly suited our duct tape strategy.

Cost Summary

• 32" Samsung Frame TV + optional frame: $600 + $100 = $700
• Image generation: ~$0.006 each
• Pi Zero W: ~$40 (free in my case)

(All CAD prices)

Altogether, not exactly a cheap build for what it does. But, now that it's running, it's less than 1 cent each time I want to "frame" some new art! And, while in my case this was an "extra" TV I added to my office for fun, if you did this on your main TV, you're just $40 away from this additional capability.

Results

Now I can simply say, "Hey Siri, FrameIt", and describe my desired image. Seconds later, it's hanging on my wall. My wife and I have been having fun surprising each other with new art - here are a few examples that I think turned out pretty well:

A Note on prompt engineering: Prompt engineering as a whole is way out of scope for this post, and I'm far from an expert. But, if you're generating AI art for the first time, just note that the prompt an make a huge difference in the quality of your output. Often, adding additional details and stylistic notes can help.

If you want to duplicate this setup, you can head over to the Github repo. If you do and you're willing to share your results, tag me on X!

Next Steps

There are a ton of ways we could improve this hacky setup - here are a few that are top of mind for me:

• Better prompting - Possibly powered by LLM prompt enhancements (example from Nous).
• Faster upscaling - Upscaling often completes in <10s, but occasionally runs for over 1 min! I'd like to consider other options.
• Image editing - The generations are a roulette wheel right now, with no ability to tweak an image that is *almost* right. You're stuck tweaking your prompt and trying again, which may not necessarily give you an especially similar image. It'd be cool to have a second command that can perform image-to-image edits instead, or an option to use a fixed seed.
• Slideshow creation - I like the art largely sitting static, but I could see a use case for generating an entire slideshow/album of images with a single prompt.
• Better image management - Cleaning up old images, revisiting an old generation, downloading a nice piece to your phone... all that would be cool (but right now I just ssh into the Pi and do so manually).