facefit
in-browser cnn · 8× smaller
sutd 60.001 applied deep learning · team of 4
the problem
people buy eyewear online that doesn’t suit their face and send it back. those returns cost money and carry a real carbon footprint from reverse logistics.
what i built
a dual-branch model: a cnn reads the portrait, a small mlp reads four facial-proportion ratios from mediapipe landmarks, and an attention gate learns when to trust pixels vs geometry. it classifies face shape (heart / oblong / oval / round / square) and recommends eyewear with a 3d try-on overlay.
the final model runs in-browser via onnx, so inference happens on the visitor’s device with no server round-trip.
stack
outcome
the final model reaches ~76% validation accuracy and holds its own against a fine-tuned resnet-18 baseline at just 1.4m parameters. live as an in-browser demo.