SW (Image) Rec

By Tyler Richards

Throughout my time at UF, I've loved to utilize a gym called Southwest Rec. Usually, I lift or play indoor soccer when I am there, but occasionally i'll play basketball. Now the problem is, I don't really want to go to the gym if there are too many people there and I can't get a good workout in. So my goal here is to figure out how to tell if the gym is actually super crowded or not.

Attempt #1: Counters

The SW Rec website has their employees walk around various areas of the gym around every 45 minutes or so and physically count the number of people there, and nicely put the raw counts online for anyone to see. See a few of the counts below.

In [46]:
from IPython.display import Image
Image(filename='sw_rec_example.png')
Out[46]:

This screenshot was taken at 11:30am, but as you can see, the counts were from nearly an hour before. In a gym that can fluctuate by 40 members in a 15 minute span, is this the best we can do? Is there a way to get a more accurate count?

Attempt #2: Live Cameras

Another section of the public facing website is called 'Live Cameras', which I went to and was pretty much immediately creeped out by.

In [53]:
Image(filename='live_cams_sw.png')
Out[53]:

Why would UF on a public wifi server have live cameras pointing at students for anyone to see
This is unbelievably creepy
WHY WOULD YOU EVER DO THIS

Let's help UF be way less creepy by automating the counter

My initial theory is that UF keeps the live cameras online because people want something more accurate than the counter. So let's figure out if we can use real time object recognition to improve the counting of the list.

YOLO: Real-Time Object Detection

We don't have time or data to collect and classify tens of thousands of pictures, so I found a system with readily available pre-trained weights for simple objects in YOLO. YOLO's paper is really cool and can be found where most interesting work is published these days, on arxiv.
Basically, YOLO is super fast because it uses a single pass at the image using a convolutional neural net and gives class probabilities directly at the pixel level. Sick.
Ok so we know it's really cool, let's give it a whirl. As with most of these things, the setup is the hardest part and setting up YOLO on your computer can be done by following this tutorial.

First we have to scrape the data from the rec sports website, which, using the urllib library, is doable.

In [48]:
import os
import urllib.request

urllib.request.urlretrieve("http://recsports.ufl.edu/cam/cam1.jpg", "data/weight_room.jpg")
Out[48]:
('data/weight_room.jpg', <http.client.HTTPMessage at 0x1145ffbe0>)

Once we do that, we can run this line which runs detect from the command line, passes the .weights that the tutorial had us download, runs the image detection on the data/weight_room.jpg, and then saves the results to a weight_results.txt file.
A lot going on in one line! The only things I changed when messing around with this line is where to output the results and the path for the picture I wanted to run YOLO on.

In [49]:
os.system('./darknet detect cfg/yolov3.cfg yolov3.weights data/weight_room.jpg out_filename > weight_results.txt')
Out[49]:
0

Results

I tested this on quite a few different scraped pictures, see the results below.

In [58]:
Image(filename="results/basketball_prediction.jpg")
Out[58]:
In [55]:
Image(filename='results/weight_room_pred.jpg')
Out[55]:
In [59]:
Image(filename="Results/activity_room.jpg")
Out[59]:

This seemed to work incredibly well, but how could recports use this instead of the counters? Let's use a bit of pandas on the results output to count the people in the frame.

In [ ]:
import pandas as pd
Algo_results = pd.read_csv("results.txt", sep=":", header=None, names = ['Item', 'Percent'], skiprows=1)
In [61]:
Algo_results.head()
Out[61]:
Item Percent
0 person 100%
1 person 99%
2 person 99%
3 person 99%
4 person 99%

Now that we have a dataframe of both the predicted item and the confidence of the detection algorithm (the threshold can be set in the cli), we can count the people and call it a day!

In [62]:
Algo_results[Algo_results['Item'] == "person"].count()[0]
Out[62]:
22

And there we go! Good enough for me.

Cool potential next steps

I would love to hook this up to my google calendar and have it text me if there are more than 30 people or so at the gym when I am supposed to go. That would be fantastic.
Another interesting idea is to scrape this data over the course of a semester, so that the following semester we could predict how many people were going to be there at a certain time. That would be particularly useful for recsports in hiring decisions, as they could optimally shift hiring around the confidence bands of the prediction.

NB

If you're reading this, and you know someone at recsports, or you work in their tech department or for the contractor they hired, please for the love of god take down those live streams. I'll help you implement this better solution!!