There are a ton of native apps that help you remove the background of an image, and after seeing the good work at remove.bg[1] I had a hunch even a script kiddie like myself could get a decent result using client side javascript with an off the shelf image segmentation model.

After a little research I found Google's MediaPipe Selfie Segmentation[2]. It's a derivative of the same tech driving Google Meet. While it is performant on mobile phones, the tradeoff[3] is a less than pixel perfect mask. It performs best when there is clear separation between the person and the background. I went down the rabbit hole in this problem space and found this[4] and this[5] which led me to believe with a little more work I can get a better result. I am completely out of my depth in ML and would most likely need a partner to help me implement better quality image segmentation for high definition photos. I considered using remove.bg's API but with the goal of keeping Headshot free, the unit economics won't work for the time being.

Headshot is built with Vue and Tailwind and hosted on Firebase. It leverages glfx.js[6] letting the user apply image filters to both the foreground and background. With a few filter tweaks you can put together a pretty decent looking profile photo. I compress uploaded images as mobile browsers seem to fall over on raw files taken from an iPhone 13. Final image compositing was pretty straightforward using HTML Canvas and the CanvasRenderingContext2D.globalCompositeOperation. I do apply a bilateral filter[7] to the segmentation mask to smooth out the edges and provide an adjustable blur[8] on the background to create a light ring effect.

The quality of the result really comes down to the quality of the starting photo. When I followed these general tips[9] for taking a good headshot I was able to create something that can pass as a reasonable headshot. I am pretty happy with the result but would love to ultimately improve the segmentation mask under more challenging photographic conditions. The lengths that the Google Meet team went to get a professional result are an interesting technical read[10] that is well above my pay grade.

I hope someone on Hacker News can use Headshot to create a great looking profile photo that they can upload to LinkedIn, Zoom, or Slack.

Happy to answer any questions.

Jason

[1] - https://www.remove.bg/

[2] - https://google.github.io/mediapipe/solutions/selfie_segmenta...

[3] - https://drive.google.com/file/d/1dCfozqknMa068vVsO2j_1FgZkW_...

[4] - https://github.com/anilsathyan7/Portrait-Segmentation

[5] - https://github.com/Volcomix/virtual-background

[6] - https://evanw.github.io/glfx.js/

[7] - https://github.com/m0ose/imageRoutines

[8] - https://github.com/flozz/StackBlur

[9] - https://headshot.page/tips

[10] - https://ai.googleblog.com/2020/10/background-features-in-goo...