To make the build of the Brick Sorting Machine (see build instructions) easy to reproduce, I decided to deliver the image based LEGO® brick identification for the machine as a remote web-service.
As a hobby project with super limited development time, creating a remote service was the most time efficient path forward. It bypasses the need for me to dive into optimizing and quantizing neural networks for AI accelerator hardware. To continuously enhance its accuracy, the classifier training learns from the images this service processes. Every image submitted will be used to improve the model’s performance.
Public Dataset Releases
After manual labeling of the data my plan is to publicly release the training data sets in regular intervals for training of everybody’s own classifier projects
SW Interface
The brick sorting machine software stack will be ready to connect to this classification backend within the next days. The service currently is in private beta test phase. If you are interested and building a compatible machine please reach out to me via Discord to get an API-KEY.
Backgrounds
The image classification backend is reachable via api.bricksortingmachine.com. Its neural network is fully tuned to work best as a classifier for the Brick Sorting Machine with its ability to capture two viewing perspectives simultaneously. It classifies parts into one of 21 part classes, which perfectly fits the machine’s 22 bins (with one bin for unclassified parts).
The classification extracts the two known regions of interest from the provided image and feeds them through the visual feature encoders. At this point this mid-level feature information is fused from both perspectives and provided jointly to the classification head. The network thus always works on image pairs both at training and inference time. This way the the classifier can make maximum use of the mutual multi-view information which often is key to uniquely identify part classes.
Currently available part classes:
bar
brick1x
brick2x
brick_modified
car
hinge
human
multiple_parts
plane
plate
plate1x
plate2x
plate_modified
plate_shaped
round
round_slope
slope
slope1x
small
tile
window
