PyGameExamplesAndAnswers

reply.it

“The first principle is that you must not fool yourself and you are the easiest person to fool.”
Richard P. Feynman

Surface and NumPy or cv2

Check pixel

Related Stack Overflow questions:

is there a faster way of checking if list[i] of pygame surface has alpha 0

Load OpenCV (cv2) image

Related Stack Overflow questions:

How do I convert an OpenCV (cv2) image (BGR and BGRA) to a pygame.Surface object

The shape attribute of a numpy.array is the number of elements in each dimension. The first element is the height, the second the width and the third the number of channels.
A pygame.Surface can be generated by pygame.image.frombuffer. The 1st argument can be a numpy.array and the 2nd argument is the format (RGB or RGBA).

Get the size (widht, height) for the pygame.Surface object by slicing:

size = cv2Image.shape[1::-1]

Determine the target format for the pygame.Surface object, depending on the third channel:

format = 'RGBA' if cv2Image.shape[2] == 4 else 'RGB'

Since the source format is BGR or BGRA, but the target format is RGB or RGBA, the red and blue channels have to be swapped:

cv2Image[:, :, [0, 2]] = cv2Image[:, :, [2, 0]]

In the case of a grayscale image, the shape of the array must be changed using numpy.reshape and the gray channel must be expanded to a red-green and blue color channel using numpy.repeat:

cv2Image = np.repeat(cv2Image.reshape(size[1], size[0], 1), 3, axis = 2)

With his data the pygame.Surface object can be generated by pygame.image.frombuffer:

surface = pygame.image.frombuffer(cv2Image.flatten(), size, format)

To ensure that the image has the same pixel format as the display Surface and for optimal performance, the Surface should be converted with either convert or convert_alpha:

surface = surface.convert_alpha() if format == 'RGBA' else surface.convert()

Complete function cv2ImageToSurface:

def cv2ImageToSurface(cv2Image):
    if cv2Image.dtype.name == 'uint16':
        cv2Image = (cv2Image / 256).astype('uint8')
    size = cv2Image.shape[1::-1]
    if len(cv2Image.shape) == 2:
        cv2Image = np.repeat(cv2Image.reshape(size[1], size[0], 1), 3, axis = 2)
        format = 'RGB'
    else:
        format = 'RGBA' if cv2Image.shape[2] == 4 else 'RGB'
        cv2Image[:, :, [0, 2]] = cv2Image[:, :, [2, 0]]
    surface = pygame.image.frombuffer(cv2Image.flatten(), size, format)
    return surface.convert_alpha() if format == 'RGBA' else surface.convert()

📁 Minimal example - Load cv2 image to PyGame Surface

Load cv2 image to PyGame _Surface_

Capture screen and convert Surface to cv2

Related Stack Overflow questions:

📁 Minimal example - cv2 image From PyGame Surface

Actually, a cv2 image is just a three-dimensional numpy array. the 1st dimension is the height, the 2nd the width and the 3rd the number of channels in the order blue, green, red.
Use pygame.surfarray.pixels3d to reference pixels into a 3d array:

img_array = numpy.array(pygame.surfarray.pixels3d(selected_area))

Unfortunately, the pixels are stored in row mayor order. The color channels are in the order red, green, blue. So you need to reformat this array. transpose the array:

image_object = numpy.transpose(img_array, (1, 0, 2))

Finally swap the red and blue color channel with either

image_object[:, :, [0, 2]] = image_object[:, :, [2, 0]]

image_object = cv2.cvtColor(image_object, cv2.COLOR_RGB2BGR)

Convert function:

def pygameSurfaceToCv2Image(mysurface, x, y, w, h):
    selected_area =  mysurface.subsurface((x, y, w, h))
    img_array = numpy.array(pygame.surfarray.pixels3d(selected_area))
    image_object = numpy.transpose(img_array, (1, 0, 2))
    #image_object[:, :, [0, 2]] = image_object[:, :, [2, 0]]
    image_object = cv2.cvtColor(image_object, cv2.COLOR_RGB2BGR)
    return image_object

Load frames from NumPy array

Related Stack Overflow questions:

Pygame and Numpy Animations

Load multiple frames (data) of 8-bit BGRA images of shape (y, x, 4, k) to a list of pygame.Surface objects:

surf_list = [pygame.image.frombuffer(d[:,:,[2, 1, 0, 3]].flatten(), (data.shape[1::-1]), 'RGBA') for d in data.transpose(3, 0, 1, 2)]

respectively

surf_list = []
for d in data.transpose(3, 0, 1, 2):
    bytes = d[:,:,[2, 1, 0, 3]].flatten()
    size = data.shape[1::-1]
    format = 'RGBA'
    surface = pygame.image.frombuffer(bytes, size, format)
    surf_list.append(surface)

Explanation:

Use numpy.traspose to bring the 3rd (frame) axis moving axis 2 to the front (see Iterating over arbitrary dimension of numpy.array) and iterate through the frames:

for d in data.transpose(3, 0, 1, 2):

Create a pygame.Surface from each frame by pygame.image.frombuffer():

surface = pygame.image.frombuffer(bytes, size, format)

pygame.image.frombuffer() has 3 arguments, bytes, size, format. bytes is the 1 dimensional byte array of pixel data. numpy.ndarray.flatten return a copy of the array collapsed into one dimension. Most likely the order of the color channels is BGRA rather than RGBA. Hence you have to swap the red and blue color channel (d[:,:,[2, 1, 0, 3]]). You can skip this if the order of the color channels is RGBA:

bytes = d[:,:,[2, 1, 0, 3]].flatten() # for BGRA

bytes = d.flatten() # for RGBA

size is a tuple with 2 elements (x, y) and specifies the size of the image. The size can be get form numpy.ndarray.shape:

size = data.shape[1::-1]

size = (data.shape[1], data.shape[0])

format specifies the image format and has to be 'RGBA' ('BGRA' doesn’t exist):

format = 'RGBA'

📁 Minimal example - Load frames from NumPy array

Pygame and Numpy Animations

Load animated GIF

Or use OpenCV/opencv-python library and VideoCapture

Write a function, that can convert a cv2 pygame.image image to a pygame.Surface a nd use the library to load a GIF frame by frame:
(see also Read GIF files in Python and How do I convert an OpenCV (cv2) image (BGR and BGRA) to a pygame.Surface object)

import cv2

def cv2ImageToSurface(cv2Image):
    size = cv2Image.shape[1::-1]
    format = 'RGBA' if cv2Image.shape[2] == 4 else 'RGB'
    cv2Image[:, :, [0, 2]] = cv2Image[:, :, [2, 0]]
    surface = pygame.image.frombuffer(cv2Image.tostring(), size, format)
    return surface.convert_alpha() if format == 'RGBA' else surface.convert()

def loadGIF(filename):
    gif = cv2.VideoCapture(filename)
    frames = []
    while True:
        ret, cv2Image = gif.read()
        if not ret:
            break
        pygameImage = cv2ImageToSurface(cv2Image)
        frames.append(pygameImage)
    return frames

📁 Minimal example - Load animated GIF PyGame Surface list using cv2

How can I load an animated GIF and get all of the individual frames in PyGame?