Is there a workable solution to the problem of slow hard disk access doing IO?

Started by Matty, January 15, 2024, 02:43:56

Previous topic - Next topic

Matty

Greetings peoples,

I've been using a simple program I wrote to capture video feed from my PC and write it to hard disk as a set of numerically named png files.

The issue I'm stuck on, that I don't know how to address is this:

Video images can be captured quite quickly, easily at a decent frame rate, that's not a problem.

Writing to disk though is slow, unless the file being written is small, really small I can't write output at more than about 12fps, which is too slow to capture video in real time.

I can create a buffer and store hundreds of full screen images in memory before writing them to disk in a separate thread. But the problem still exists that the longer the video is recorded, the greater the gap becomes between where the video capture frame is up to, and where the disk output is up to, until eventually available memory is used up by that difference.

I can't see an obvious solution.

To use an analogy it's like this:
Two runners are running a 42km marathon.  Runner 1 (the screen capture) runs at 20kph. Runner 2 (the writing to disk) runs at 5kph.  The longer they run for, the greater the gap in their position in the race.  Is it an impossible scenario? Is it impossible for the 5kph runner to keep up with the 20kph runner for the duration of the race, or at least stay within a sufficiently small length behind that it's not a problem for the officials waiting for the race to end?

blinkok

Why not just write it to disk as a video file and then grab the frames later

Matty

Because I want the uncompressed frames first.

In any case - I found a solution: multiple worker threads each writing separate frames to disk.

It basically works as follows:

thread captures screen image in 'blocks of 4' frames at a time.
four other threads write each of the images 1-4 frames to disk each taking one of the frames to write.
next capture of next four frames doesn't begin until previous four have all been written to disk.

It seems to work okay. I can record video at about 30fps in real time on my old PC using this technique with a program written in Java. Next up is to record the audio as well.