Designing the data flow of a Ping Pong Tracker on a Embedded Device

At the Rochester Institute of Technology, I am currently a senior in Computer Engineering Technology. As a final step before graduation, we have one class with one task, design a ping pong ball tracker using an embedded device.

To solve this task we’ve created a group of five engineers with different internship experiences and skills. Two FPGA Engineers, two Software Engineers, and one Hardware Engineer teamed up to build this project. In this team, our roles were separated not in terms of our engineering disciplines but more of responsibilities such as project management, communications between services, testing, FPGA design, and the user interface. My role is to develop the communications between services and create a user interface to display the field in three dimensions.

The project restrictions are to use a Snickerdoodle board with a stereo-vision camera system attached to it and to display the position in a 3D environment. The ball should be indicated if it is in or out of the field.

As we started to develop the project, we noticed that there might be a problem with running everything on the CPU. We have 1GB in RAM and having to run the whole code as such would be tough without optimizing. That is why the team set off to find ways to increase the speed using the FPGA.

To solve this, the team found a blob detector to process an image and find centroids of blobs. This is important as we have much more computing available when working towards a solution. Using this circuitry, the throughput of data increases as the rate of processing is drastically faster.

As we have more compute power available and increased throughput, the design of the data flow has changed slightly. The goal for our team is to increase the experience of the gameplay. We decided to not use the given tools, like Unity, and we decided to use Three.js which is an amazing library for 3D visualizations. Initially, the plan was to use React to build the visualization but since the complexity is much less for the view and interactions, we went with plain HTML and vanilla JS.

Here is a basic view of the field we built:

This interface connects to a Pub/Sub service. This will allow implementing updates to the position by messages sent to these connected interfaces. Another data point to be sent to the interface is whether the ball is in or out of the physical field.

From the FPGA to the interface the data flow was accomplished by: reading the register addresses, sending them to Kafka, and reading the queue from the proxy server to send to subscribers. All of these steps will be containerized and isolated in a user-created docker network. Running a sample service sending a circular path at 1-second intervals, we were at about 75% of the memory available to us. This was tested on a Raspberry Pi 3B with the same specs as our requirement device.

There is a high-risk, high-reward situation with this design as we do not know if the blob detector will work fully as we want. This is because we need to convert it to use two camera feeds and the processing of centroids to get depth values. To overcome this risk our backup plan consists of a change in where the image is processed.

For the backup, only initial processing and the overlay generation portion of the blob detector will be used. Then the centroid detection is done in the CPU, or the algorithm service we will have. We will have to find a separate alternative to Kafka to spare as much compute power as possible. The publisher-subscriber design will still exist. An alternative message broker to use is ZeroMQ which is much less resource intensive than Kafka, and also we do not have an extreme amount of data to be queued.

Each of these services will be dockerized and run using docker-compose to easily run them on other Snickerdoodle devices if needed.

More updates on this post will come soon.