You spent hours in communion with your computer editing together little Tammy Tanglefoot's dance recital. You have reduced forty minutes of lurches and stumbles into five minutes of graceful ballet. Next, you print your masterpiece to tape or CD ROM and sit back to view the results.
Whoops, what's that...? You view the shot again and notice a tiny jump in the picture (as opposed to a grand jete for young Tanglefoot). You view the shot again. Yep, it's still there. What in the world happened? You view some more of your show. Whoops, there's another one.
These little jumps and herky jerky motion are not Tammy's fault; they are the result of dropped frames. Normally, video runs at about 30 frames (pictures) per second. This constant uninterrupted flow of pictures creates smooth motion. If one or more pictures are missing from this series (dropped frames), you'll notice a jerkiness to the motion (unless it's a picture of something standing still, in which case you don't notice any missing pictures). Dropped frames are a dead giveaway that something in your computer system is misbehaving, and is probably overloaded.
What causes dropped frames -
This question has a general answer and 101 specific answers, some of which involve more computer savvy than most video editors (myself included) possess. Let's start with the general answer.
Video involves a lot of data. Raw, uncompressed video consumes over 20 megabytes per second, which is a lot to store and also a lot to pass through your computer on its way to somewhere else such as a hard drive, or video tape machine. Then there is audio which in its uncompressed (raw) state adds about 20% more data to the stream. Only the most advanced computers and software can handle noncompressed video data. The rest of us use data which is compressed somewhat reducing the rate to perhaps 3 megabytes per second. Whatever the case, something in your system cannot keep up. Perhaps the compressor isn't compressing fast enough, dropping a frame every once in a while. Perhaps your computer is too slow (maybe 700 MHz as opposed to 1.3GHz). Perhaps the disk drive is too slow, unable to record all the data as it comes in, even if it is compressed first. Remember the I Love Lucy show where Lucy and Ethel worked on an assembly line putting chocolates into a box? Sometimes the conveyor belt moved faster than Lucy could and she missed a chocolate. This is what happens when a computer system cannot keep up with the video and audio data; it skips some of it, often dropping a frame.
Compression and dropped frames -
If you are starting with raw analog audio and video, you need to digitize it before passing it to your computer. (If you are using a DV camcorder with an IEEE1394 (Firewire or i.Link) connection, the digitizing and some compression have already been done in the camera; your digitizing card is not involved except to forward the data to the hard drive). The digitized audio and video signals result in about 20 megabytes per second of data which, unless you have a professional video editing workstation, have to be compressed to about 3 megabytes per second. Rates higher than this will probably result in dropped frames or lost image information.
There are several ways to compress the video data. DV, MPEG-1 , MPEG-2, MJPEG, Wavelets, Indio, Cinepak, and similar methods of video compression mathematically look at the picture and remove data redundances and image details that your eye is unlikely to notice. Indio, Cinepak, and other methods of compression involve capturing video at low resolution such as 352 x 240 pixels (1/4 screen size and also ¼ the normal TV screen resolution of 704 x 480 pixels). Video can also be captured at a resolution of 176 x 120 (1/16 normal screen). Further, video can be captured at a rate of 15fps (frames per second) as opposed to the normal 30fps. Although these latter techniques might be satisfactory for CD ROM and internet film clips, the average videographer and DVD producer will want the full 30 fps and 704 x 480 screen resolution and will therefore use DV, MPEG1, MPEG2, MJPEG, or Wavelets as his compression method.
Compressors come in two varieties: hardware and software. In a software compressor, your computer's brain is programmed to do the compression and decompression jobs. Although this process uses up a lot of your computer's mental resources, this is not really a problem for any fast computer with lots of memory. If your system's CPU is running at less than 700 MHz, it may not (depending on circumstances) be able to keep up and will drop frames. Also, if you are working with minimal memory (say under 500 MB of RAM) or your motherboard has a slow bus speed (under 100 MHz), these weaknesses may all conspire to drop a frame or two.
This is where a hardware codec (compression/decompression) circuit board comes to the rescue. Here the hardware codec does all of the number crunching for the compression, leaving your computer resources untouched. Also the hardware codec is optimized to do the job in a timely fashion. In short, if you have a modern, fast computer with plenty of resources, then a software codec is unlikely to overtax the machine. If you have an older, slower computer, then a hardware codec is better equipped for the job.
DV is the type of compression performed by digital camcorders (DV and Digital-8) as they record their tape. It automatically removes some color resolution, then compresses the data 5:1. Camcorders do an excellent job of digitizing and compressing data. DV's 5:1 compression looks about the same as MJPEG's less efficient 3:1 compression.
MJPEG (Motion Joint Picture Experts Group) compression is used in many high end professional digitizers and editors. The amount of compression is variable allowing the user to use minimal (such as 3:1) compression for very high video quality. Since the data is not reduced very much, it fills up the drives quickly (about 30 gigabytes per hour) and places high demand on the codec, computer, and drives. MJPEG at 5:1 compression gives good video quality with fewer demands on the computer system. A compression ratio of 10:1 makes a fairly crummy picture with minimal demands on the computer and drives.
MPEG-1 (Motion Picture Expert Group Level 1) has a high compression rate, placing the least demands on your computer and drives, but captures video only at a 352 x 240 resolution (30 frames per second). This lower video quality is fine for publishing video on CD ROMs and the Web, two cases where the data must be kept to a minimum.
MPEG-2 is another compression scheme aimed at higher quality video and is used on some professional systems.
MPEG video is very difficult to edit with frame accuracy. MPEG uses a small number of "true" frames as a reference, and inbetween them places artificial frames containing only the differences between the complete frames. The complete video frames are called I frames and the artificial frames are called P (for predicted) and B (for bidirectional frames). Professional MPEG2 editing machines create only I frames so that the video can be edited at any frame. Once the editing job is done, the data is reprocessed and about 90% of the data hungry I frames are replaced with data stingy P and B frames. The resulting output is very highly compressed. Professional workstations need to be used for this process because it takes a lot of resources to create the MPEG stream of I frames (when the clip is all made of I frames) involves a lot of data moving quickly. Once converted to MPEG with I, P, and B frames, there is not much data and nearly any computer can handle it.
Incidentally, even MPEG made with I, P, and B frames can be edited, it just requires backflips to be edited accurately (you can cut on only one of every seven frames, for instance). In short, using MPEG with I frames, you need a fast machine, that handles a lot of data. If roughly editing I, P, and B MPEG, a slower computer will do.
There are editors that will take an MPEG-2 stream and reconstitute it into all I frames around the edit points and during transitions, then perform the edit. It requires fast computers and professional software to perform elegantly and in real time.
How much compression should you use when you capture your raw footage? If you use too much compression, the image will look blocky. If you use too little compression, the data will overcome your computer system and you'll drop frames. If your capture software allows you to set the data rate, you should pick a number, capture something with a lot of motion, then see if any frames were dropped. If none were, then reduce the compression and try the experiment again. Continue until you start dropping frames, then you will know the minimum compression (maximum data rate) your machine can handle.
Dropped frames is not your only concern. You may wish to employ a higher compression rate than your machine requires just to reduce the amount of disk space used by your video clips. One way to choose the amount of compression needed for a video clip is to consider the source of the video and how the video will be used. If the source were professional Betacam and the edited results were going back to Betacam, then, if your machine can handle it, a 3:1 compression might be appropriate. If you were shooting and laying back to home VHS, then a compression of 9:1 may be appropriate. If you have no idea at all, I suggest a compression ratio of 7:1 which results in a data rate of 2.5 Mbs and will store an hour on a 10 gigabyte drive.
Hard drives -
Hard drives come in three types, IDE, RAID, and SCSI. The RAID and SCSI drives are very fast and are found on professional machines. They seldom drop frames unless deluged with uncompressed or HD video. IDE drives, common on home PCs, work at various speeds and generally require video to be compressed before the data is slow enough for them to handle.
IDE drives come in two types, ATA/33 and the newer Ultra ATA/66. The numbers here represent how MANY megabytes per second can be fed to the drive through its controller circuitry. As you might expect, 66 is twice as fast as 33. Neither of these numbers represent the speed of the drive itself. Although many drives list the rpm (how fast they spin in revolutions per minute)--- and the faster, the better--- this number is less important than the access time and the sustained data rate. Access time tells how long it takes for the drive to go from one spot to the next. A 10 microsecond access time is considered adequate for video. The sustained data rate tells how fast the drive can absorb or pour out data without interruption. A 1 megabyte per second sustained data rate will work for highly compressed video, while a 4 megabyte per second data rate will handle DV (the signals that come from a DV camcorder precompressed). Seagate, IBM, Maxtor, Quantum, and Western Digital offer inexpensive Ultra ATA/66 hard drives well suited for DVediting.
RAID (Redundant Array of Independent Disks) represents a team of disk drives that work together to provide more disk space and speed. It used to be that only SCSI drives could be configured as a RAID, but now even IDE drives can be converted to a RAID or faster speed and greater memory. Promise Technology's $150 FastTrack66 RAID controllers can make 2-4 IDE drives into a RAID capable of 20 MBs, fast enough to handle uncompressed video without dropping frames.
Things you can do to minimize dropped frames -
I presumed you read the instruction manual that came with your capture card (ha, that's a good one!). It may tell you how to avoid dropped frames. You also know to read the instructions that came with any other part of your editing software. Now I know these statements sound as tired as the airline instructions on how to fasten your seat belt, but you'd be surprised at how many people don't look in the obvious places for the solutions to their problems. For those of you whose manuals offered gibberish in places where answers were supposed to be found, here are some possible solutions to your dropped frames problem:
1. If possible, dedicate one drive to your computer system (all programs, including your editing software) and a second drive for your video clips. Sometimes your computer looks to the hard drive for instructions on how to do something, and this process will interrupt the smooth flow of video data from the drive. If you use a separate drive, one of them can be busy serving the computer's needs while the other is unfettered with such housekeeping and able to concentrate on playing video.
If you cannot do the above, then at least partition your drive into two parts: the first (drive C:) would hold the computer programs, while the second (drive D:) would be set aside for video clips. Organized this way, at least the computer can find the video material faster than if it is mixed in with the programs. Incidentally,if one drive is faster than the other, put your video on the fastest one.
2. Defragment your video hard drive before each new job. In the process of editing, bits of video clips get scattered everywhere. If your drive is trying to play them, and read heads have to dance all over the platters trying to find these bits, the process will take time and will drop frames. If the disk is defragmented, then the video material will be contiguous and easy for the drive heads to locate. To defragment a drive in Windows 95, for instance, you select START> ACCESSORIES >SYSTEM TOOLS>DISK DEFRAGMENTER and then select the drive you wish to defragment. Defragment this drive/partition before making any important video captures and before you render your finished movie.
3. Make sure your drive is media rated or AV rated. Some computer drives, every 10 minutes or so, briefly interrupt their data transfers to calibrate themselves. This never bothers your computer's management of general data, but wreaks havoc with a computer that is trying to record an uninterrupted flow of video data or play it back. AV rated drives do not cost appreciably more, but their reliable high data rates will save you a lot of dropped frames.
4. The hard drive you use for video should be connected to your computer's primary IDE channel (unless you are using SCSI). Your computer has two IDE channels. The primary channel is faster than the secondary channel. Each channel can handle a master and a slave hard drive. Video should go on the slave drive. Make your boot drive the master drive on the primary channel. (If you're not deep into computer mechanics, ask your local computer guru, or any 10 year old to help you do this. It isn't hard.) Put your CD, DVD, zip, and other drives on the secondary IDE channel.
5. Record your video on the outside portion of the platter. How do you do this? The computer generally starts recording from the outside and works in towards the middle of the platter. If you start with a cleanly erased drive, then your new material will be recorded on the faster, outside tracks. When the drive gets full and is recording to the inside tracks, the data does not record or read off as fast. The same is true for CD ROMs and DVDs; the outside tracks move data faster.
If you find that the beginning of your video captures are working fine but later on you are dropping frames, it is probably because you have filled up the fast tracks on the disks and are now working on the slower inside tracks. One solution may be to increase your compression rate so that it is more than necessary when you start filling your disk, and is still adequate when you finish filling your disk. Most video capture cards display dropped frames data as they operate so that you can stop the capture and reduce the data rate if necessary before the capture is complete.
Another trick that works with some systems such as the Mirovideo DC30+, is to reserve your video capture file size before you start capturing any video. That way your capture will be at the fastest place on your hard drive. When you are finished capturing, you can save your file in some other directory (it no longer needs to be in the fastest place).
6. If you have transitions/effects or rendered portions of your video, try to edit those first. This will place the rendered temporary files on the faster outside edges of the hard drive platters. This provides the transitions with the higher data rates they need.
7. When capturing, editing, and playing back video, shut down all of the other, unneeded programs. Disable screen savers, e-mail, fax/modem software, wallpaper, anti-virus programs, and anything else sitting in your system tray (those tiny icons at the bottom right edge of your computer screen) that saps your computer's attention. Some virus protection works in the background and can interfere with editing. Disconnect your PC from any networks (unless it's a high performance video workstation designed for networked editing). Also turn off energy saving options. This way your computer can give its full attention to its video tasks. As an example, if your computer were running at its max trying to capture video and the screen saver suddenly clicked on, the computer's resources would be distracted for a moment installing the screen saver, and would drop a frame in the process. The same is true if your computer checks for e-mail messages while in the midst of a video capture.
8. Locate your swap file (virtual memory) on the same drive as your video. Having a swap file in the C: drive while capturing to the D: drive can cause you to drop frames. Evidence that your swap file is in the wrong place: while capturing or playing back video from the D: drive you notice that the C: drive is being accessed; it should be quiet.
9. Make sure you are using the latest drivers (software that tells your computer how to operate hardware devices such as capture cards, audio cards, graphics adapters, etc.). If you are running Windows XP, make sure you have installed the Windows XP drivers for your hardware if they are available. Check with your board and software manufacturers to see if they have upgraded drivers for all your hardware. This also goes for your audio and video cards.
10. Place your temp files and scratch disk on your video drive along with your video projects.
11. Make sure your editing software uses the same data rate used when you captured your raw footage. This consistent data rate will also improve the speed at which your transitions will render.
12. Make sure your video capture card, editing software, and Windows use the same settings for audio quality (ie. 44,100 Hz, 16 bit stereo). To do this in Windows, you would go to START >CONTROL PANEL >MULTIMEDIA >AUDIO>CUSTOMIZE > 44,100 Hz, 16 BIT STEREO to make this adjustment.
13. Using Adobe Premier, don't select "Optimize stills in Key frame and Rendering Options" in the compression dialogue box of both the Project >Settings and Export > movie > settings dialogue boxes. If the "optimized stills" option is unavailable in Premier 4.0.1, move the "Make Flattened" plug-in file from the Additional Plug-Ins folder to the Adobe Premier Plug-Ins folder, then restart Premier. If you are using Premier 4.X for Windows, deselect "Optimize Stills" in the Compression Settings dialog box. Why do you have to make this seemingly backwards adjustment? In order to reduce rendering time, Premier 3.0 or later for Mac and Premier 4.X or later for Windows switch how they process information; video clips are done one way and still images are done another way. Many MJPEG capture cards have difficulty switching between one method and the other, resulting in dropped frames. By switching "optimized stills" off, you force Premier to render every frame of the still image just like it does with the video clips, a process that may take longer but doesn't shift gears midstream.
For more Premier/Windows tips, go to http://www.ccgnv.net/admitone/.
14. Start capturing video a few moments before the important footage begins. Many capture cards drop a few frames at the beginning of the capture process when the camcorder or video player is coming up to speed. You can trim off the excess footage in the editing process.
15. Avoid the digitizing/compressing process altogether and feed pre-digitized compressed data via IEEE 1394 from a DV camcorder into your computer. This takes the frame-dropping computer codec out of the equation. When editing is completed, feed the data back to the camcorder via IEEE 1394 for recording, again bypassing any codec in the computer. Incidentally, you get better IEEE 1394 support if you run Windows XP, 98 or Windows 98SE or Windows 2000 (vs Windows 95), but don't use the Windows generic IEEE 1394 driver; use the driver that comes with the IEEE 1394 card you install --- it'll drop fewer frames.
16. Use your NLE strictly for editing video. Remove video games and other software that may change your computer's settings if used. Also, optimize your CMOS/BIOS settings, if you know how.
17. If you have an old or cheap display adapter (video card), turn it down to 16 bit. Modern fast display cards with plenty of RAM are able to make pictures without using much of the computer's resources. Older cards, however, rely on the computer's memory and bog down when making pictures. By turning the adapter down to 16 bits, the colors won't look as pretty on your screen, but the video card won't suck as much vitality out of your computer causing you to drop frames. You make this adjustment by going to START>SETTINGS>CONTROL PANEL>DISPLAY>SETTINGS and selecting "high color (16 bit)" on the color palette. While you are there, you may wish to switch to the screen saver tab and select "none".