following, unedited, was posted by Cary Milsap of Method-R
to comp.databases.oracle.server on 3-Aug-2004. It says much of what needs to be
said ... and very succinctly. Only new technology such as Apple's Xserve RAID
changes what has been said below.
My argument is this:
1. Mirroring is excellent for resilience. Striping is excellent for performance (as long as the striping is done with large enough granularity
as not to introduce a concurrency problem). Combining the two technologies makes for excellent resilience and excellent performance.
2. But mirroring is expensive per byte of storage. Hence RAID levels 2, 3, 4, 5, and 6 were proposed. Their design goals were to lessen the
expense-per-byte of storage of RAID level 1 (mirroring). For example, with G=5 RAID level 5, the price of resilience per byte of storage is 5/4 of a
4-disk array instead of 8/4.
3. But the additional parity operations required to run RAID  produce a performance penalty. The
performance penalty is so bad (4:1 for many common Oracle operations) that it begged for a solution: namely, the
introduction of non-volatile cache and a complicated kernel (more lines of source code than Oracle7.1) to run it all. So, as long as the cache is big
enough to stay ahead of your sustained throughput, a RAID level 5 array can
perform almost as well as a well-configured array that uses levels 0 and 1 combined.
4. But the RAID level 5 array is deficient in a number of important ways (performance degradation during partial outage as Howard mentioned, worse
availability performance, worse read performance [contrary to popular
belief], inability to take online backups with a simple re-silver operation, etc.). And the cache and microcode make the RAID level 5 array more
expensive than the 0+1 or 1+0 array, which defeats the whole purpose of inventing RAID level 5 in the first place.
I'm not telling you not to use RAID level 5 (we use it for low-throughput systems with flexible availability
requirements). Just don't assume that it's cheaper than using RAID levels 0 and 1 for high-throughput systems. The
insidious thing is that your initial procurement cost is probably cheaper if you use RAID level 5, but if you don't figure out in your initial capacity
plan that "disk sizing" means counting I/O calls, not counting bytes of storage, then you're in for a lot more expense than you thought, if you go
the RAID level 5 route.
By the way, did you notice in the original Chen/Katz/Gibson/Patterson/Lee paper that block-striped RAID level 1 is mathematically identical to RAID
level 5 if you set G=2? So, in a sense, I'm a big proponent of RAID level 5. I just don't like the idea, for high throughput systems, of using parity
group sizes bigger than 2 or buying the cache and microkernel.