mbpool(9)

NAME

mbpool - buffer pools for network interfaces

SYNOPSIS

#include <sys/types.h>
#include <machine/bus.h>
#include <sys/mbpool.h>
struct mbpool;
int
mbp_create(struct   mbpool   **mbp,   const   char    *name,
bus_dma_tag_t dmat,
        u_int    max_pages,    size_t    page_size,   size_t
chunk_size);
void
mbp_destroy(struct mbpool *mbp);
void *
mbp_alloc(struct mbpool *mbp, bus_addr_t *pa, uint32_t *hp);
void
mbp_free(struct mbpool *mbp, void *p);
void
mbp_ext_free(void *, void *);
void
mbp_card_free(struct mbpool *mbp);
void
mbp_count(struct  mbpool  *mbp,  u_int  *used,  u_int *card,
u_int *free);
void *
mbp_get(struct mbpool *mbp, uint32_t h);
void *
mbp_get_keep(struct mbpool *mbp, uint32_t h);
void
mbp_sync(struct mbpool *mbp,  uint32_t  h,  bus_addr_t  off,
bus_size_t len,
        u_int op);
MODULE_DEPEND(your_module, libmbpool, 1, 1, 1);
options LIBMBPOOL

DESCRIPTION

Mbuf pools are intended to help drivers for interface cards

that need
huge amounts of receive buffers, and additionally provides a

mapping
between these buffers and 32-bit handles.

An example of these cards are the Fore/Marconi ForeRunnerHE

cards. These
employ up to 8 receive groups, each with two buffer pools,

each of which
can contain up to 8192. This gives a total maximum number

of more than
100000 buffers. Even with a more moderate configuration the

card eats
several thousand buffers. Each of these buffers must be

mapped for DMA.
While for machines without an IOMMU and with lesser than

4GByte memory
this is not a problem, for other machines this may quickly

eat up all
available IOMMU address space and/or bounce buffers. On

sparc64, the
default I/O page size is 16k, so mapping a simple mbuf

wastes 31/32 of
the address space.

Another problem with most of these cards is that they sup

port putting a
32-bit handle into the buffer descriptor together with the

physical
address. This handle is reflected back to the driver when

the buffer is
filled, and assists the driver in finding the buffer in host

memory. For
32-bit machines, the virtual address of the buffer is usual

ly used as the
handle. This does not work for 64-bit machines for obvious

reasons, so a
mapping is needed between these handles and the buffers.

This mapping
should be possible without searching lists and the like.

An mbuf pool overcomes both problems by allocating DMA-able

memory page
wise with a per-pool configurable page size. Each page is

divided into a
number of equally-sized chunks, the last MBPOOL_TRAILER_SIZE

of which are
used by the pool code (4 bytes). The rest of each chunk is

usable as a
buffer. There is a per-pool limit on pages that will be al

located.

Additionally, the code manages two flags for each buffer:

``on-card'' and
``used''. A buffer may be in one of three states:

free None of the flags is set.

on-card Both flags are set. The buffer is assumed to be

handed over to

the card and waiting to be filled.

used The buffer was returned by the card and is now

travelling

through the system.

A pool is created with mbp_create(). This call specifies a

DMA tag dmat
to be used to create and map the memory pages via

bus_dmamem_alloc(9).
The chunk_size includes the pool overhead. It means that to

get buffers
for 5 ATM cells (240 bytes), a chunk size of 256 should be

specified.
This results in 12 unused bytes between the buffer, and the

pool overhead
of four byte. The total maximum number of buffers in a pool

is max_pages
* (page_size / chunk_size). The maximum value for max_pages

is 2^14-1
(16383) and the maximum of page_size / chunk_size is 2^9

(512). If the
call is successful, a pointer to a newly allocated struct

mbpool is set
into the variable pointed to by mpb.

A pool is destroyed with mbp_destroy(). This frees all

pages and the
pool structure itself. If compiled with DIAGNOSTICS, the

code checks
that all buffers are free. If not, a warning message is is

sued to the
console.

A buffer is allocated with mbp_alloc(). This returns the

virtual address
of the buffer and stores the physical address into the vari

able pointed
to by pa. The handle is stored into the variable pointed to

by hp. The
two most significant bits and the 7 least significant bits

of the handle
are unused by the pool code and may be used by the caller.

These are
automatically stripped when passing a handle to one of the

other functions. If a buffer cannot be allocated (either because the

maximum number of pages is reached, no memory is available or the memo

ry cannot be
mapped), NULL is returned. If a buffer could be allocated,

it is in the
``on-card'' state.

When the buffer is returned by the card, the driver calls

mbp_get() with
the handle. This function returns the virtual address of

the buffer and
clears the ``on-card'' bit. The buffer is now in the

``used'' state.
The function mbp_get_keep() differs from mbp_get() in that

it does not
clear the ``on-card'' bit. This can be used for buffers

that are
returned ``partially'' by the card.

A buffer is freed by calling mbp_free() with the virtual ad

dress of the
buffer. This clears the ``used'' bit, and puts the buffer

on the free
list of the pool. Note that free buffers are NOT returned

to the system.
The function mbp_ext_free() can be given to m_extadd() as

the free function. The user argument must be the pointer to the pool.

Before using the contents of a buffer returned by the card,

the driver
must call mbp_sync() with the appropriate parameters. This

results in a
call to bus_dmamap_sync(9) for the buffer.

All buffers in the pool that are currently in the ``on

card'' state can
be freed with a call to mbp_card_free(). This may be called

by the
driver when it stops the interface. Buffers in the ``used''

state are
not freed by this call.

For debugging it is possible to call mbp_count(). This re

turns the number of buffers in the ``used'' and ``on-card'' states and

the number of
buffers on the free list.

SEE ALSO

mbuf(9)

CAVEATS

The function mbp_sync() is currently a no-op because

bus_dmamap_sync(9)
is missing the offset and length parameters.

AUTHORS

Harti Brandt <harti@FreeBSD.org>

BSD July 15, 2003

BSD