Hi Terence, I haven't played around with OpenCV in a while, but it should be perfectly possible to get this working. You might have a look at pyglet/media/sources/avbin.py to see if there are any techniques you could use. Also, instead of using a thread for fetching the stream data, why not use pyglet.clock to schedule that?
Let me know how you get on with this. I don't have a video device right now to try this myself, but a nice OpenCV example might be nice to have in the repo. On Friday, June 30, 2017 at 6:31:10 AM UTC+9, Terence Liu wrote: > > Hi all, I just started graphics programming with pyglet. opencv's > VideoCapture stream outputs BGR numpy arrays, and most online > tutorials/solutions use array.tostring() or array.tobyte() to let > pyglet.image.ImageData construct the texture. It works but the conversion > is relatively slow ~30 ms per frame. Then I started trying out ctypes > arrays: > > import pyglet > import numpy as np > import cv2 > from threading import Thread > > stream = cv2.VideoCapture(0) > stream.set(cv2.CAP_PROP_FPS, 30) > _, frame_raw = stream.read() > SCREEN_HEIGHT, SCREEN_WIDTH, NCOLOR = frame_raw.shape > window = pyglet.window.Window(SCREEN_WIDTH, SCREEN_HEIGHT) > fps_display = pyglet.clock.ClockDisplay() > > def get_from_stream(): > global frame_raw > while True: > _, frame_raw = stream.read() > > thread_get_from_stream = Thread(target=get_from_stream) > thread_get_from_stream.daemon = True > thread_get_from_stream.start() > > @window.event > def on_draw(): > window.clear() > # image = pyglet.image.ImageData(SCREEN_WIDTH, SCREEN_HEIGHT, 'BGR', > frame_raw[:, ::-1, :].tobytes(), -SCREEN_WIDTH * 3) # flip horizontally > image = pyglet.image.ImageData(SCREEN_WIDTH, SCREEN_HEIGHT, 'BGR', > np.ctypeslib.as_ctypes(frame_raw[:, ::-1, :].ravel()), -SCREEN_WIDTH * 3) # > flip as well > > # image = pyglet.image.ImageData(SCREEN_WIDTH, > SCREEN_HEIGHT, 'BGR', np.ctypeslib.as_ctypes(frame_raw[:, ::-1, > :].copy().ravel()), -SCREEN_WIDTH * 3) # copy and flip > > # image = pyglet.image.ImageData(SCREEN_WIDTH, SCREEN_HEIGHT, 'BGR', > np.ctypeslib.as_ctypes(frame_raw.ravel()), -SCREEN_WIDTH * 3) # no flip > image.blit(0, 0) > > fps_display.draw() > > pyglet.clock.schedule_interval(lambda x: None, 1/30.) > pyglet.app.run() > > See the four lines in on_draw(). THe first line with frame_raw[:, ::-1, > :].tobytes() works at ~20FPS. What's strange about the second is that there > is a lot of flickering alternating between black screen and images, and > during the flickering some images are flipped, some are not! I thought this > has anything to do with how views are seen at lower levels, but when I use > the third line with a copy(), the situation is the same! Then when I use > the fourth line, there is no strange flipping issues but flickering still > persists. What's going on? > -- You received this message because you are subscribed to the Google Groups "pyglet-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/pyglet-users. For more options, visit https://groups.google.com/d/optout.
