Subject: Re: [vserver] Sponsoring VServer getting up to date with mainline
From: "Michael S. Zick" <mszick@morethan.org>
Date: Fri, 22 Aug 2008 18:15:23 -0500

On Fri August 22 2008 17:52, Ed W wrote:
> Michael S. Zick wrote:
> >
> > The true advantage of a hardware card, is you only read/write
> > the storage data once over the PCIe link.
> >   
> 
> 
> I agree in theory, but I have been desperately disappointed with a two 
> 3Ware cards of various generations.  They struggled to get read or write 
> performance to match a half that of a single drive in a 4 drive raid5 
> config.  
>

It is very easy to build a card in a 16x form factor, perhaps using only
a single (1x) channel;  These cards would be Bragging Rights and/or 
Marketing Rights only cards.

Ah, but building a 16x form factor card with electronics that can run that
fast - - much harder, also much too expensive.

> Some of the limitations looked very much like those of the PCI  
> bus and indeed 3wares benchmarks suggested that you needed faster than 
> PCI to get >20-40MB/sec out of their cards (ugg).  
>

The PCI bus is too slow, even the double width, double speed version.
Even a 2x card can run into bandwidth limitations.
Keep in mind, PCI is a half-duplex, shared, bus.

The 16x and 32x PCIe is driven directly by the bridge chipset, no PCI
bus involved getting to/from RAM.  At least, not if your pocket book
is deep enough.
This is another reason why it is hard to find a motherboard with more
than one 16x or 32x slot (in transfer width, not just connector size).

If Herbert wants fast compiles, he should keep his kernel tree in a
RAM disk on a 32 or 64Gbyte ram machine.

That would probably cut him down to a single serving of his favorite
beverage per compile.

Mike
> To me this looked  
> very much like the data was being transferred too many times versus 
> theory (also I discovered later that nearly all these hardware cards 
> can't do xor fast enough to keep up with a sensible modern disk, hence 
> software raid5 can often be faster because you aren't lagging on the xor 
> engine...)
> 
> I think the safe benchmark is all in software and be very cautious of 
> these hardware SATA cards.  That said, the various scsi cards I have 
> tried absolutely scream.  Why there are only cr*p sata hardware cards I 
> just don't understand?
> 
> Oh well
> 
> Ed W
> 
>