OpenCV + Python + Webcam: Create a Simple Game (Avoid the falling logo)

What will we cover in this tutorial?

Is it a bird? Is it a plain? No, it is falling objects from the sky.

With OpenCV you can get a stream of frames from your webcam. Process the data in Python to create an easy prototype of a simple game, where you should avoid the falling objects. We will cover how to built that in this tutorial.

Step 1: The game explained

The game is quite simple and is built based on a few ideas.

  1. Setup a live stream from your webcam.
  2. Insert falling objects starting from the top of the frame at a random vertical position.
  3. If the object hits you (the player) you get subtracted 1 point in your score.
  4. On the other hand, if you avoid the object and it hits the bottom of the frame without hitting you, you gain one point in your score.
  5. Play until bored or tired.

Step 2: How can this game be built easy?

You basically need three components to create this game.

Firstly, we need a way to take and process each frame from the webcam. That is, create a live stream that can show you what is happening in the view of the frame. This will only require a way to read a frame from the webcam and show it on the screen. If this is done repeatedly, you have a live stream.

Secondly, something that can make objects that fall down in your frame from a random position. That is, it should remember where the object was in the last frame and insert in a new, lower position in the new frame.

Thirdly, something that can detect where you are in the frame. Hence, if the object and you are in the same position. You are subtracted a point from your score and a new object is created at the top.

The great news is that we can make all the simple by using Python.

Step 3: Get a live stream from a webcam

a thing that is required as a prior condition for something else to happen or exist.

Google meaning

A simple stream from the webcam can be created be the following code.

import cv2


# Capture the webcam
cap = cv2.VideoCapture(0)
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 640)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 480)


while True:
    # Get a frame
    _, frame = cap.read()

    # Update the frame in the window
    cv2.imshow("Webcam", frame)

    # Check if q is pressed, terminate if so
    if cv2.waitKey(1) == ord('q'):
        break

# Release the webcam and destroy windows
cap.release()
cv2.destroyAllWindows()

If you have troubles installing OpenCV, please read this tutorial. You might need to change the width and height of the webcam. You can find all the possible resolutions your webcam support by following this tutorial. The reason to lower the resolution is to increase the processing time and not slow the game.

Another approach, where you can keep the full resolution, is to only resize the images you make the processing on.

Step 4: Motion detection

The idea behind a simple motion detector is to have a picture of the background. Then for each frame you will subtract the background from the frame. This will identify all new object in the frame.

To get a good picture of the background it might be an idea to let the webcam film for a few frames, as it often needs to adjust.

The idea is mapped out here.

import cv2
import numpy as np

# Capture the webcam
cap = cv2.VideoCapture(0)
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 640)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 480)

# To capture the background - take a few iterations to stabilize view
while True:
    # Get the next frame
    _, bg_frame = cap.read()
    bg_frame = cv2.flip(bg_frame, 1)

    # Update the frame in the window
    cv2.imshow("Webcam", bg_frame)

    # Check if q is pressed, terminate if so
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Processing of frames are done in gray
bg_gray = cv2.cvtColor(bg_frame, cv2.COLOR_BGR2GRAY)
# We blur it to minimize reaction to small details
bg_gray = cv2.GaussianBlur(bg_gray, (5, 5), 0)


# This is where the game loop starts
while True:
    # Get the next frame
    _, frame = cap.read()
    frame = cv2.flip(frame, 1)

    # Processing of frames are done in gray
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    # We blur it to minimize reaction to small details
    gray = cv2.GaussianBlur(gray, (5, 5), 0)

    # Get the difference from last_frame
    delta_frame = cv2.absdiff(bg_gray, gray)
    # Have some threshold on what is enough movement
    thresh = cv2.threshold(delta_frame, 100, 255, cv2.THRESH_BINARY)[1]
    # This dilates with two iterations
    thresh = cv2.dilate(thresh, None, iterations=2)
    cv2.imshow("track", thresh)

   # Update the frame in the window
    cv2.imshow("Webcam", frame)

    # Check if q is pressed, terminate if so
    if cv2.waitKey(1) == ord('q'):
        break

# Release the webcam and destroy windows
cap.release()
cv2.destroyAllWindows()

First let the background be as you want to (empty without you in it). Then press q, to capture the background image used to subtract in the second loop.

The output of the second loop could look similar to this (Maybe with you instead of me).

Output

For a more detailed explanation of a motion tracker, you can read this tutorial on how to make a motion detector.

Step 5: Adding falling objects

This will be done by inserting an object in our frame and simply moving it downwards frame-by-frame.

To have all functionality related to the object I made an object class Object.

The full code is here.

import cv2
import numpy as np

# Capture the webcam
cap = cv2.VideoCapture(0)
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 640)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 480)

# To capture the background - take a few iterations to stabilize view
while True:
    # Get the next frame
    _, bg_frame = cap.read()
    bg_frame = cv2.flip(bg_frame, 1)

    # Update the frame in the window
    cv2.imshow("Webcam", bg_frame)

    # Check if q is pressed, terminate if so
    if cv2.waitKey(1) and 0xFF == ord('q'):
        break

# Processing of frames are done in gray
bg_gray = cv2.cvtColor(bg_frame, cv2.COLOR_BGR2GRAY)
# We blur it to minimize reaction to small details
bg_gray = cv2.GaussianBlur(bg_gray, (5, 5), 0)


# Read the logo to use later
class Object:
    def __init__(self, size=50):
        self.logo_org = cv2.imread('logo.png')
        self.size = size
        self.logo = cv2.resize(self.logo_org, (size, size))
        img2gray = cv2.cvtColor(self.logo, cv2.COLOR_BGR2GRAY)
        _, logo_mask = cv2.threshold(img2gray, 1, 255, cv2.THRESH_BINARY)
        self.logo_mask = logo_mask
        self.speed = 15
        self.x = 100
        self.y = 0
        self.score = 0

    def insert_object(self, frame):
        roi = frame[self.y:self.y + self.size, self.x:self.x + self.size]
        roi[np.where(self.logo_mask)] = 0
        roi += self.logo

    def update_position(self, tresh):
        height, width = tresh.shape
        self.y += self.speed
        if self.y + self.size > height:
            self.y = 0
            self.x = np.random.randint(0, width - self.size - 1)
            self.score += 1

        # Check for collision
        roi = tresh[self.y:self.y + self.size, self.x:self.x + self.size]
        check = np.any(roi[np.where(self.logo_mask)])
        if check:
            self.score -= 1
            self.y = 0
            self.x = np.random.randint(0, width - self.size - 1)
            # self.speed += 1
        return check


# Let's create the object that will fall from the sky
obj = Object()

# This is where the game loop starts
while True:
    # Get the next frame
    _, frame = cap.read()
    frame = cv2.flip(frame, 1)

    # Processing of frames are done in gray
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    # We blur it to minimize reaction to small details
    gray = cv2.GaussianBlur(gray, (5, 5), 0)

    # Get the difference from last_frame
    delta_frame = cv2.absdiff(bg_gray, gray)
    # Have some threshold on what is enough movement
    thresh = cv2.threshold(delta_frame, 100, 255, cv2.THRESH_BINARY)[1]
    # This dilates with two iterations
    thresh = cv2.dilate(thresh, None, iterations=2)
    # cv2.imshow("track", thresh)

    hit = obj.update_position(thresh)
    obj.insert_object(frame)

    # To make the screen white when you get hit
    if hit:
        frame[:, :, :] = 255

    text = f"Score: {obj.score}"
    cv2.putText(frame, text, (10, 20), cv2.FONT_HERSHEY_PLAIN, 2, (0, 255, 0), 2)
    # Update the frame in the window
    cv2.imshow("Webcam", frame)

    # Check if q is pressed, terminate if so
    if cv2.waitKey(1) == ord('q'):
        break

# Release the webcam and destroy windows
cap.release()
cv2.destroyAllWindows()

But does it work?

Trying the game

No I am not too old for that stuff. Or, maybe I was just having one of those days.

It works, but of course it could be improved in many ways.

Leave a Reply