FTDI isochronous USB transfer - Page 2

Do you have a question? Post it now! No Registration Necessary

Translate This Thread From English to

Threaded View
Re: FTDI isochronous USB transfer
Hello,

Please read above:

g9u5dd43 snipped-for-privacy@yahoo.com wrote in message
Quoted text here. Click to load it

I set up some code like this:

1) Global variable (from a link in this thread):
CCircularBuffer< char, 14880000 > circBuffer;

2) Working thread's function:
UINT CTestMThreadDlg::USBCommThread(LPVOID pParam)
{
    CTestMThreadDlg* pTestMThread = (CTestMThreadDlg*) pParam;

    FT_STATUS ftStatus;
    DWORD eventDWord;
    DWORD rxBytes;
    DWORD txBytes;
    DWORD bytesReceived;
    char rxBuffer[63488];
    unsigned int dataReceived = 0;

    HANDLE hEvent = CreateEvent(
        NULL,
        false, // auto-reset event
        true, // non-signalled state
        "");
    // Check the value of the hEvent!!!
    if (hEvent == NULL) {
        DWORD lastError = GetLastError();
        printf("Error %d when tried to create handle\n", lastError);
    }
    else {
        DWORD eventMask = FT_EVENT_RXCHAR;
        ftStatus = FT_SetEventNotification(eventMask, hEvent);

        while (dataReceived < 148800) {
            WaitForSingleObject(hEvent, INFINITE);

            ftStatus  = FT_GetStatus(&rxBytes, &txBytes,
                &eventDWord);
            
            if (ftStatus == FT_OK && (eventDWord & FT_EVENT_RXCHAR)
                && rxBytes > 0) {
                ftStatus = FT_Read(rxBuffer, rxBytes,
                    &bytesReceived);
                circBuffer.Write(rxBuffer, rxBytes);
                dataReceived += rxBytes;
            }
        }
    }

    return 0;
}

I need/want to fill the complete buffer, or a buffer of that size
(14.88MB, or bigger!!!). My questions/problems that I am facing are:

1) Why if I pass the "false" value to the third parameter (initial
signaling state) of the CreateEvent function, the
WaitForSingleObject(hEvent, INFINITE); function NEVER returns?????? I
had to put it in _initial signaled_ state for it to return ("true"
value), but the documentation and examples here provided say something
very different. The other way, using the "false" value, is to use
WaitForSingleObject(hEvent, XXX), with XXX any timeout (I tried with
500, and worked).

2) (And the most important). I need to gather 14.88MB thru USB. In
order to do so, I have a counter of all the bytes received, which
increments every time I read data and write it to Circular Buffer. The
reading process continues until I have read all the data that I need.
The problem is that if I loop reading for more than 148800 bytes in
"while (dataReceived < 148800)", the thread seems to get blocked....
Why???? I've tried with values 2*148800, 1488000, and similar
(>148800) but it is always the same. It is worth saying that this
behaviour happens regardless of what I do in question 1)

All of your comments and recommendations are VERY welcome.

BTW, I've liked the fact that you also send the replies to my e-mail,
because that way I can have the feedback quicker (it takes a long time
in www.deja.com, where I post them).

Thank you very much,

JaaC

Quoted text here. Click to load it

Re: FTDI isochronous USB transfer
Hello,

----- Original Message -----
Sent: Thursday, November 20, 2003 9:13 PM
Subject: Re: FTDI isochronous USB transfer


Quoted text here. Click to load it

Yes, returns OK.

Quoted text here. Click to load it

Why forever? When do you finish receiving data?

Quoted text here. Click to load it

This is no the case, because (for testing purposes) the DSP goes into an
infinite loop, sending 1488 bytes every 26 msecs. I even tried with a
smaller "packet size", say 372 bytes each 26 msecs. I still get the same
amount of data in the PC side, altough it takes a bit longer, of course (I
haven't measured, though). I expect the PC side to stop "listening" to the
peripheral, when it gathers enough amount of data... I could also stop
sending data, but this is no the case, because I haven't even got the data
that I need!!!!

Quoted text here. Click to load it

I can't believe that it is no possible to send 1488 bytes every 26
msecs!!!!! Could you suggest a mechanism for reading the data even
faster???? The call that triggers the data-collecting thread is this:

CWinThread* pCommsThread = AfxBeginThread(USBCommThread, this,
THREAD_PRIORITY_TIME_CRITICAL);

I think it's the right way, isn't it?

Quoted text here. Click to load it

I will ask you a BIG favor, which I think won't be very problematic for you.
As you have much more experience, and also have some material (DLP FTDI-PIC
board) to try with, couldn't you please write a small application on which
you send 1488 bytes from the PIC to the PC, in a loop with, say, 100.000
iterations (total transfer: 1488 x 100.000 = 148800000 Bytes = 14.88MB)????
This is the problem that I have at hand, and which I need to solve... maybe
the problem that I have at this moment also arises for you, and we can find
a different approach to solve it. I know that you are VERY busy.... I am
just asking for a favor, if you want to help. As you can see, I've tried
every imaginable approach...

Thank you very much,

JaaC

Re: FTDI isochronous USB transfer
snipped-for-privacy@ieee.org says..
Quoted text here. Click to load it


I'll repeat my advice from many posts back in this thread:

Get rid of the threads.  Code your USB receiving routine in the
main program loop and get it working there.  When you have the
USB data transfer working when it has the complete attention of
the PC,  THEN you can think about adding the overhead of thread
management.

I regularly manage to transfer multi-megabyte files from a
microcontroller system to the PC at rates of up to 200KBytes
per second (limited by the CF card read speed on the microcontroller).
Your application should work just fine if it has the PC
resources it needs.  Start by giving it ALL the PC resources
then work your way up to a more sophisticated application
model.

DO NOT try to output nice debugging messages to the screen
while you are receiving the data.   You may get into
parts of the display driver that block waiting for a
screen refresh or some such thing.  (who,  outside NVIDIA,
really knows what happens when you write to the screen these
days??)  Simply count the bytes and display the total
after a fixed period or byte count.
Quoted text here. Click to load it


Mark Borgerson



Re: FTDI isochronous USB transfer
Hello,

Quoted text here. Click to load it

That's what I tried the very first time, without success. That's the
way I supposed I should work. Anyway, everywhere else I've read, I've
found that it is done with threads. And now, they don't seem to be the
problem for me. The problem seems to be deeper...

Quoted text here. Click to load it

THIS IS _THE_ PROBLEM:

After very much trying, I've found that regardless of the packet size,
regardless of the time between packets, regardless of the value passed
in the FT_SetUSBParameters (read and write timeouts) and regardless of
the value of the Latency Timeouts, all the data I can get is exactly
262144 bytes, which equals exactly four times 65536 (2^16)....
WHY?????? If I try to wait for more data than that, the application
never returns, the thread (or the app itself) gets
blocked!!!!!!!!!!!!!!

Quoted text here. Click to load it

... and I'm not using them anymore, that' not the problem!!!!!!!

Thanks,

JaaC

Re: FTDI isochronous USB transfer
snipped-for-privacy@ieee.org says...
Quoted text here. Click to load it

If you can't get it working without threads,  I doubt that
you'll be able to get it working WITH threads.


Quoted text here. Click to load it

Are you certain that the transmitter is still trying to send data
at that point? Are you monitoring the progress in the transmitting
end to be sure that it is not at fault?  How does it respond if
the PC is unable to accept data for some interval?
Quoted text here. Click to load it


Mark Borgerson


Re: FTDI isochronous USB transfer (Finally done!!!!)
Hello,

Quoted text here. Click to load it

You were right, Mark.

It was my mistake, a quite stupid mistake from my part, I must admit
it. At that point the DSP stopped writting data to the USB FIFO. Let
me explain why, to share the experience, as I promised:

I use the USB FIFO to write data from the DSP to the PC in the most
independet possible way. I mean, I don't want much intrussion from the
trasferrring process into the DSP-tasks performed by the DSP, so to
speak. In that order of ideas, I use DMA to transfer the data from the
DSP to the PC. This is the scenario: every 26 mSecs, I set up a DMA
process, and fire it away, so the DSP can continue doing number
crunching while the on-chip I/O processor sends the data to the PC,
thru the USB FIFO. There is an acknowledge signal, which I connect to
the TXE signal in the FIFO for "handshaking". I also set up some
waitstates in order to meet the timing requeriments of the FIFO, which
is somewhat slower than the DSP.

For the DMA process you must set up basically these parameters:
1) Internal memory
- Base address of internal buffer (where the internal DMA data buffer
starts)
- Length of internal buffer (how long is it the buffer)
- Stride (the "step" at which the internal DMA pointer increments)

2) External memory
- Base address of external buffer (keep in mind that the FIFO is a
memory-mapped peripheral, so this is the address of the FIFO)
- Length of external buffer (how long is it the buffer)
- Stride (the "step" at which the external DMA pointer increments)

My mistake was that I had a non-zero external stride AND I WAS NOT
UPDATING THE BASE ADDRESS OF EXTERNAL BUFFER, so, after a few (262144,
to be precise) writes to the FIFO, the address at which it was
attached was not being decoded, so I was not writting the data to the
USB FIFO, as you (Mark) suggested!!!!! The solution was to update the
address of external buffer with every DMA.

Other solutions could have been:
1) Lengh of external buffer = 1
2) External stride = 0

For the PC side, I am using this approach (notice the much bigger
value of 1500000 bytes I used for testing, and worked):

Actually, what the 2nd thread does is to check how much data have I
received from the USB FIFO, and stop when I have collected enough
data.

void CTestMThreadDlg::OnReadUsb()
{
    // Create thread for data reading
    CWinThread* pCommsThread = AfxBeginThread(USBCommThread, this,
THREAD_PRIORITY_TIME_CRITICAL);
    // Create thead for data parsing
    CWinThread* pParseThread = AfxBeginThread(ParseThread, this);
}

UINT CTestMThreadDlg::USBCommThread(LPVOID pParam)
{
    CTestMThreadDlg* pTestMThread = (CTestMThreadDlg*) pParam;

    FT_STATUS ftStatus;
    DWORD eventDWord;
    DWORD rxBytes;
    DWORD txBytes;
    DWORD bytesReceived;
    char rxBuffer[65536];

    HANDLE hEvent = CreateEvent(
        NULL,
        false, // auto-reset event
        true,  // signalled state
        "");
    // Check the value of the hEvent!!!
    if (hEvent == NULL) {
        DWORD lastError = GetLastError();
        AfxMessageBox("Error %d when tried to create handle\n", lastError);
        return -1;
    }
    else {
        DWORD eventMask = FT_EVENT_RXCHAR;
        ftStatus = FT_SetEventNotification(eventMask, hEvent);

        if (ftStatus == FT_OK) {
            gettingData = true;
            while (gettingData) {
                WaitForSingleObject(hEvent, INFINITE);

                ftStatus  = FT_GetStatus(&rxBytes, &txBytes, &eventDWord);
                
                if (ftStatus == FT_OK && (eventDWord & FT_EVENT_RXCHAR) && rxBytes
Quoted text here. Click to load it
                    ftStatus = FT_Read(rxBuffer, rxBytes, &bytesReceived);
                    circBuffer.Write(rxBuffer, bytesReceived);
                    dataReceived += bytesReceived;
                }
            }
            AfxMessageBox("USB Comm thread terminated");
            return 0;
        }
        else {
            AfxMessageBox("Error setting notification\n");
            return -1;
        }
    }
}

UINT CTestMThreadDlg::ParseThread(LPVOID pParam)
{
    while (dataReceived < 1500000) {
    }
    CString str;
    str.Format("%d byes on memory!", dataReceived);
    AfxMessageBox(str);
    gettingData = false;
    dataReceived = 0;

    AfxMessageBox("Parse thread terminated");
    
    return 0;
}

I've found the two-thread scheme particularly useful because that way
the main application doesn't get blocked by the USB-data-waiting
process, and I can stop it if I want.

Of course, it still needs some polishing, but at least I have the USB
side of the application done!!!

I want to thank all of you guys who helped me to find a solution for
this problem, thank you _for your patience_ and your support.

And yes, FTDI parts are great for USB projects!!!!!

Regards,

JaaC


Quoted text here. Click to load it

Re: FTDI isochronous USB transfer (Finally done!!!!)
snipped-for-privacy@ieee.org says...
Quoted text here. Click to load it


<<SNIP>>



Glad to see you figured it out.  Now that you've got it working
with threads on the PC side,  I'll keep a copy of that code around
in case I ever get back to a PC application that needs threads.

Mark Borgerson


Re: FTDI isochronous USB transfer
snipped-for-privacy@ieee.org says...
Quoted text here. Click to load it

So don't sleep.  If you get no data,  you don't write anything to the
output and try again.  There's nothing wrong with that..  The essence of
any polling operation is that you often expect to get no data.  The
sleep helps() only if other threads need a lot of processing time.
Quoted text here. Click to load it
You don't really need synchronization.  You need to get all the data.
If the PC gets ahead and often finds there is no data to read (which
will happen because the USB sender sends at intervals of 1mSec), that
is not a problem.
Quoted text here. Click to load it


Mark Borgerson

Re: FTDI isochronous USB transfer
On 14 Nov 2003 14:23:16 -0800, snipped-for-privacy@ieee.org (Jaime Andres

Quoted text here. Click to load it

check ftStatus to see if it is successful, since you are not declaring
the device handle, maybe it doens't default to zero like the others.
ftStatus = FT_SetEventNotification(ftHandle,EventMask,hEvent);


Quoted text here. Click to load it
You'd be better off dumping all the data you collect into a
pre-allocated array, and then dump the array after you test period is
complete.  Writing formated strings to a disk file ain't all that
quick.

Quoted text here. Click to load it

Don't sleep when you are polling.  Sleep allows the thread/process to
be suspended to allow other tasks to run. I use a higher priority.
j = SetThreadPriority( ThreadID, THREAD_PRIORITY_TIME_CRITICAL );


Quoted text here. Click to load it

  Huh?  Why do you have th synchronized the packets?

Quoted text here. Click to load it

 Possiblely the event is not defined.  Test the return parameter.

Quoted text here. Click to load it

The USB communication is typically delay because its just getting up
to speed.


Re: FTDI isochronous USB transfer
Hello,

My comments are inlined, please read them (and reply back, of
course!):

Quoted text here. Click to load it

I am using a global ftHandle, previously allocated and (succesfully)
open. In other words, my FT_SetEventNotification (without ftHandle)
calls the _real_ FT_SetEventNotification with the global ftHandle as a
parameter. And returns FT_OK (0), as expected.

Quoted text here. Click to load it

Good idea. Mine was a first approach to check what was going on,
PACKET BY PACKET.

Quoted text here. Click to load it

Will see... I guess the thread you talk about is the one for data
transfer only, am I right?

Quoted text here. Click to load it

Because the data I send to the PC has some kind of header, and stuff
like that. I mean, I must make sure that the transfer collects all the
data from the beginning of the first packet, not from somewhere in any
packet an continue... I mean, all the packets should be complete!

Quoted text here. Click to load it

What should I expect?

Quoted text here. Click to load it

Thanks. I'll try your suggestions, which are quite interesting, and
will let the group know how does it go.

Kindest regards,

JaaC

Site Timeline