View source1 Background and motivation
In my first year at IIT Bombay, I used the Seasons of Code programme to learn machine learning through a live application. The project was a webcam-based sign recognizer, chosen because it required a complete path from data collection to prediction. A model trained in a notebook would be only one part of the work; the output also needed to remain usable while a person moved their hands in front of a camera.
2 Data and preprocessing
MediaPipe extracts hand landmarks from webcam frames, and preprocessing converts those coordinates into a normalized feature representation. Normalization reduces the influence of hand position, scale, and camera distance so that the classifier can focus on the landmark arrangement. OpenCV provides the video-processing loop, with NumPy supporting the feature operations.
The repository separates data collection, model training, and live testing into scripts. This division makes the training inputs and inference pipeline easier to compare. If preprocessing changes during live use but not during collection or training, the model receives a different representation from the one it learned, even though the feature dimensions may still match.
3 Model and live inference
A TensorFlow/Keras model maps the feature vector to a sign prediction. In the live loop, each frame passes through landmark extraction, normalization, and inference before the predicted label is displayed. Temporal smoothing reduces rapid changes between labels caused by individual noisy frames, making the visible output less unstable.
Smoothing introduces its own tradeoff: requiring agreement across time can make a display steadier while delaying recognition of a genuine change. The live prototype therefore exposed concerns beyond the evaluation score, including missing landmarks, partial hands, motion, and how the interface should behave when its input is uncertain.
4 Evaluation and scope
The prototype achieved approximately 97% accuracy on its evaluation split, as recorded in the project description. That result applies to the collected dataset and split rather than every signer or environment. The application recognizes the signs represented in its training data; it should not be described as unrestricted sign-language translation. The project’s main educational value was connecting collection, preprocessing, training, evaluation, and live inference in one working system.