Subject: Re: [vserver] Sponsoring VServer getting up to date with mainline
From: "Michael S. Zick" <mszick@morethan.org>
Date: Fri, 22 Aug 2008 17:35:06 -0500

On Fri August 22 2008 16:53, Herbert Poetzl wrote:
> On Fri, Aug 22, 2008 at 11:45:55PM +0200, Herbert Poetzl wrote:
> > On Fri, Aug 22, 2008 at 09:58:05PM +0100, Ed W wrote:
> > > Daniel Hokka Zakrisson wrote:
> > > >
> > > >For this, I use Supermicro's AOC-SAT2-MV8. It's an 8-port card that costs 
> > > >less than 100
> > > >USD, though it does require a PCI-X slot. Granted it's got a Marvell 
> > > >chipset, but I
> > > >haven't had any problems with it.
> > > >
> > > >  
> > > 
> > > But who still has a PCI-X slot...  (I struggled just to find a 
> > > motherboard with a sensible amount of PCI slots in recently to upgrade 
> > > my media PC which needs a bunch of PCI tv cards in it! grr)
> > > 
> > > I think PCI-X is going to be close to saturated also?  It's only 1TB/s I 
> > > think?  Given a couple of badly behaved devices chewing interrupts then 
> > > I wonder how close you can get to theoretical given that you probably 
> > > pull 60-100MB per drive these days? (straight line speed)
> > 
> > to be fair, an x16 PCIe has 'only' 4TB/s, but

A different measure than others, the 16x is 16 channels, each 
channel is 250Mbyte/second, full duplex, as an 8-in-10 encoding.
Signalling protocol will reduce the actual payload to somewhat
less than that.  

The sticker here, is that 16x and 32x slots are primarily used for
high end graphics cards, resulting in it being hard to find a
motherboard with more than one of them.

Or, anything other than a graphic's card to plug into them.
Note: Per the spec, just because the card fits does not mean
it uses all 16 channels, it might only use 1x, 2x, 4x or 8x.

Still, even a modest 4Gbyte/second, full duplex, transfer rate
is respectable, and will challenge the bandwidth of other parts, 
such as the memory path.
Finding a motherboard that can absorb a 4 or 8Gbyte per second,
full duplex, bus master I/O without significantly impacting 
instruction/data fetch by the processor to the system RAM ...
You do want to avoid hardware "wait states" don't you?

The true advantage of a hardware card, is you only read/write
the storage data once over the PCIe link.

Mike
> > the problem is more that it seems to be going
> > away (as you noted) and finding a proper board
> > with two of those slots is non-trivial ...
> 
> GB not TB of course ... but TB would be cool! :)
> 
> > anyway, I think I don't want to revamp the
> > entire system ... switching the controller is
> > more than enough for now :)
> > 
> > best,
> > Herbert
> > 
> > > Ed W
> 
>