Adding my own L3 protocol in embedded Linux system with add_dev_pack

Do you have a question? Post it now! No Registration Necessary

Translate This Thread From English to

Threaded View

I'm working on high speed data transmission protocol aimed to receive data
from low resource FPGA connected to the Ethernet PHY.
The protocol is supposed to work on embedded Linux PC with high speed CPU
and NIC. The protocol will be used only in a private network, and multiple
FPGA systems will be connected via an Ethernet switch to single data
collecting embedded system.
All design should be price optimised, and therefore I need to focus
on low resource FPGAs and standard network equipment.

To keep thing as simple as possible, I have decided to get rid of the whole
IP layer and implement it as Layer 3 protocol implemented as near to the
network device driver as possible.
Due to operation only in a private network, the protocol may use randomly
chosen not used (or at least not widely used) Ethernet type code.

I've already asked some questions regarding this implementation, and got
some feedback (thread: /
), but unfortunately I had to postpone my work for some time.

Now it seems, that the problem is almost solved:
1. Using the dev_add_pack function I can register my packet handler, which
is called very quickly after the NIC receives the packet.
2. In my packet handler I can verify integrity of the packet, copy the data
to the appropriate buffer, and generate the acknowledge packet.

However it is not clear to me, whether it is safe to call the dev_queue_xmit
from the handler registered by dev_add_pack.
Another question is - how to mark my packet as handled. In the ipx_rcv
(which is also a handler registered via dev_add_pack) the misdirected packets
are simply freed by call to kfree_skb. Should I do the same after my packet
is handled?

I'd appreciate any further suggestions.
TIA & Regards,

Success ! Re: Adding my own L3 protocol in embedded Linux system with add_dev_pack
Quoted text here. Click to load it /
Quoted text here. Click to load it

I have succeeded to implement the very simple protocol with fast acknowledge
of received packets in Layer 3.
I'll publish the code under the GPL license after some cleanup (please
consider the code fragments below to be under GPL as well).
Below are the most important parts. I'd appreciate any remarks and

The acknowledge packet is created by copying first 10 (MY_ACK_COPIED) bytes of
received packet and padding it to the desired length (MY_ACK_LEN) with 0xa5

Please note, that the code below is just "proof of the concept", so it lacks
some significant error checks.

My protocol handler is installed in the following way (in the module
or when opening the device handling the protocol):

and deinstalled in the following way (in the module exit function or when
the device handling the protocol):

The protocol handler is implemented in the following way:

#define MY_ACK_LEN 64
#define MY_ACK_COPIED 10

static int my_proto_rcv(struct sk_buff * skb, struct net_device * dev, struct
packet_type * pt,
                        struct net_device * orig_dev)
    struct sk_buff *newskb = NULL;
    struct ethhdr * rcv_hdr = NULL;
    unsigned int head;
    unsigned int tail;
    int len;
    unsigned int buf_free = 0;
    char * my_data = NULL;
    unsigned long flags;
    //Copy received data to the circular buffer in kernel space
    //(it is mmapped to the user space)
    spin_lock_irqsave(&my_buf.lock, flags);
    head = my_buf.head;
    tail = my_buf.tail;
    spin_unlock_irqrestore(&my_buf.lock, flags);
    //Calculate the number of free space in the buffer
    buf_free=( tail - head -1 ) % MY_BUF_LEN;
    len = skb->len;
    if (buf_free > len) {
        //Copy the data and send acknowledge only if we can copy the received
        int res = 0;
        //Copy the data to the circular buffer, considering the possibility,
        //that the buffer will wrap around
        int bytes_to_copy_at_end = min(len,MY_BUF_LEN - head);
        int bytes_to_copy_at_begining = len - bytes_to_copy_at_end;
        if (bytes_to_copy_at_end) {
            //We use skb_copy_bits in case if the received packet contains
            //segments (is it possible?)
            res = skb_copy_bits(skb,0,&my_buf.buffer[head],bytes_to_copy_at_end);
        if (bytes_to_copy_at_begining) {
            res =
        //Update the head position
        spin_lock_irqsave(&my_buf.lock, flags);
        my_buf.head = (head+len) % MY_BUF_LEN;
        spin_unlock_irqrestore(&my_buf.lock, flags);
        //Now send the acknowledge packet
        newskb = alloc_skb(LL_ALLOCATED_SPACE(dev)+MY_ACK_LEN, GFP_ATOMIC);
        newskb->dev = dev;
        newskb->protocol = htons(0xfade);
        //Extract the MAC header from the received packet
        //Build the MAC header for the new packet
        // Based on +*/net/ipv4/arp.c#L586
< 0)
            goto error; //Well other potentially failing functions should
                        //be protected with error checks as well... To be done!
        //Copy the begining of the received packet to the acknowledge packet
        my_data = skb_put(newskb,MY_ACK_COPIED);
        res = skb_copy_bits(skb,0,my_data,MY_ACK_COPIED);
        my_data = skb_put(newskb,MY_ACK_LEN -MY_ACK_COPIED);
        //Pad the packet to the desired length
        memset(my_data,0xa5,MY_ACK_LEN - MY_ACK_COPIED);
    return 0;
    } else {
        // No place for data from the buffer
        my_buf.err_flag = 1;
    return 0;

The circular buffer is declared in the following way:
#define MY_BUF_LEN (1<<17)
struct circ_buf
    spinlock_t lock;
    volatile int head;
    volatile int tail;
    unsigned char * buffer;
    char err_flag;
} my_buf;

initialized as below:
    my_buf.head = 0;
    my_buf.tail = 0;
    my_buf.buffer = vmalloc_user(MY_BUF_LEN);
    my_buf.err_flag = 0;

and mmapped to the user space:

int my_proto1_mmap(struct file *filp,
                   struct vm_area_struct *vma)
    unsigned long vsize = vma->vm_end - vma->vm_start;
    unsigned long psize = MY_BUF_LEN;
    if (vsize>psize)
        return -EINVAL;
    remap_vmalloc_range(vma,my_buf.buffer, 0);
    if (vma->vm_ops)
        return -EINVAL; //It should never happen...
    vma->vm_ops = &my_proto1_vm_ops;
    my_proto1_vma_open(vma); //No open(vma) was called, we have called it
    return 0;

The circular buffer head, tail, err_flag are available from the user space
using the ioctl method (as in my high speed usart code:
"Lite high speed synchronous mode driver for USART in Atmel AT91SAM9260" )

HTH & Regards,

Site Timeline