FTDI isochronous USB transfer

Isn't what you can "read the USB data and stuff it into the buffer as fast as it is available" exactly whay the FT_Read function does????? I already have it!

Let's see if I understood well: I read any amount of data I have in the buffer, within a high priority thread. How do I signal to the other (the "parsing") thread that I got data in the "communications" thread???

I say "any amount of data" because if I send data from the DSP to the DSP before triggering the "communications" thread in the PC side, I'll read more than what I expected, and maybe will have some incomplete packets... So, if first start the PC application, and then start to send data to the PC, the event will be in a non-signalled state, until it receives data from the DSP, and all the packets will be complete. And once I receive the data, I should store it as soon as possible, in a (circular) buffer. One question is: why circular? Now that I know that I'll receive complete packets (am I sure of this?), wouldn't it be easier to store the received information (a full packet, niether more, nor less) into a bigger buffer of packets? The question that arises here is (again): how to signal another thread that I have a packet in memory!!

It all depends on the fact that the WaitForSingleObject(hEvent, INFINITE) function ever returns... but what if it never returns, as it seems to be my case???

Thanks you for your comments,

JaaC

Only if you call FT_Read often enough that you get lots of "no Data Available" returns.

The receiving thread should set a global variable that indicates the number of bytes in the receive buffer. The parsing thread moves the data from the receive buffer to an analysis buffer when there is more than two frames worth of data----to be sure that one complete frame is in the parsing buffer.

Parse, save, and repeat as required.

Mark Borgerson

Hello,

Please read above:

I set up some code like this:

1) Global variable (from a link in this thread): CCircularBuffer< char, 14880000 > circBuffer; 2) Working thread's function: UINT CTestMThreadDlg::USBCommThread(LPVOID pParam) { CTestMThreadDlg* pTestMThread = (CTestMThreadDlg*) pParam;

FT_STATUS ftStatus; DWORD eventDWord; DWORD rxBytes; DWORD txBytes; DWORD bytesReceived; char rxBuffer[63488]; unsigned int dataReceived = 0;

HANDLE hEvent = CreateEvent( NULL, false, // auto-reset event true, // non-signalled state ""); // Check the value of the hEvent!!! if (hEvent == NULL) { DWORD lastError = GetLastError(); printf("Error %d when tried to create handle\n", lastError); } else { DWORD eventMask = FT_EVENT_RXCHAR; ftStatus = FT_SetEventNotification(eventMask, hEvent);

while (dataReceived < 148800) { WaitForSingleObject(hEvent, INFINITE);

ftStatus = FT_GetStatus(&rxBytes, &txBytes, &eventDWord); if (ftStatus == FT_OK && (eventDWord & FT_EVENT_RXCHAR) && rxBytes > 0) { ftStatus = FT_Read(rxBuffer, rxBytes, &bytesReceived); circBuffer.Write(rxBuffer, rxBytes); dataReceived += rxBytes; } } }

return 0; }

I need/want to fill the complete buffer, or a buffer of that size (14.88MB, or bigger!!!). My questions/problems that I am facing are:

1) Why if I pass the "false" value to the third parameter (initial signaling state) of the CreateEvent function, the WaitForSingleObject(hEvent, INFINITE); function NEVER returns?????? I had to put it in _initial signaled_ state for it to return ("true" value), but the documentation and examples here provided say something very different. The other way, using the "false" value, is to use WaitForSingleObject(hEvent, XXX), with XXX any timeout (I tried with 500, and worked).

2) (And the most important). I need to gather 14.88MB thru USB. In order to do so, I have a counter of all the bytes received, which increments every time I read data and write it to Circular Buffer. The reading process continues until I have read all the data that I need. The problem is that if I loop reading for more than 148800 bytes in "while (dataReceived < 148800)", the thread seems to get blocked.... Why???? I've tried with values 2*148800, 1488000, and similar (>148800) but it is always the same. It is worth saying that this behaviour happens regardless of what I do in question 1)

All of your comments and recommendations are VERY welcome.

BTW, I've liked the fact that you also send the replies to my e-mail, because that way I can have the feedback quicker (it takes a long time in

where I post them).

Thank you very much,

JaaC

Hello,

----- Original Message ----- From: To: Jaime Andres Aranguren Cardona Sent: Thursday, November 20, 2003 9:13 PM Subject: Re: FTDI isochronous USB transfer

Yes, returns OK.

Why forever? When do you finish receiving data?

This is no the case, because (for testing purposes) the DSP goes into an infinite loop, sending 1488 bytes every 26 msecs. I even tried with a smaller "packet size", say 372 bytes each 26 msecs. I still get the same amount of data in the PC side, altough it takes a bit longer, of course (I haven't measured, though). I expect the PC side to stop "listening" to the peripheral, when it gathers enough amount of data... I could also stop sending data, but this is no the case, because I haven't even got the data that I need!!!!

I can't believe that it is no possible to send 1488 bytes every 26 msecs!!!!! Could you suggest a mechanism for reading the data even faster???? The call that triggers the data-collecting thread is this:

CWinThread* pCommsThread = AfxBeginThread(USBCommThread, this, THREAD_PRIORITY_TIME_CRITICAL);

I think it's the right way, isn't it?

I will ask you a BIG favor, which I think won't be very problematic for you. As you have much more experience, and also have some material (DLP FTDI-PIC board) to try with, couldn't you please write a small application on which you send 1488 bytes from the PIC to the PC, in a loop with, say, 100.000 iterations (total transfer: 1488 x 100.000 = 148800000 Bytes = 14.88MB)???? This is the problem that I have at hand, and which I need to solve... maybe the problem that I have at this moment also arises for you, and we can find a different approach to solve it. I know that you are VERY busy.... I am just asking for a favor, if you want to help. As you can see, I've tried every imaginable approach...

Thank you very much,

JaaC

In article , snipped-for-privacy@ieee.org says..

I'll repeat my advice from many posts back in this thread:

Get rid of the threads. Code your USB receiving routine in the main program loop and get it working there. When you have the USB data transfer working when it has the complete attention of the PC, THEN you can think about adding the overhead of thread management.

I regularly manage to transfer multi-megabyte files from a microcontroller system to the PC at rates of up to 200KBytes per second (limited by the CF card read speed on the microcontroller). Your application should work just fine if it has the PC resources it needs. Start by giving it ALL the PC resources then work your way up to a more sophisticated application model.

DO NOT try to output nice debugging messages to the screen while you are receiving the data. You may get into parts of the display driver that block waiting for a screen refresh or some such thing. (who, outside NVIDIA, really knows what happens when you write to the screen these days??) Simply count the bytes and display the total after a fixed period or byte count.

Mark Borgerson

Hello,

That's what I tried the very first time, without success. That's the way I supposed I should work. Anyway, everywhere else I've read, I've found that it is done with threads. And now, they don't seem to be the problem for me. The problem seems to be deeper...

THIS IS _THE_ PROBLEM:

After very much trying, I've found that regardless of the packet size, regardless of the time between packets, regardless of the value passed in the FT_SetUSBParameters (read and write timeouts) and regardless of the value of the Latency Timeouts, all the data I can get is exactly

262144 bytes, which equals exactly four times 65536 (2^16).... WHY?????? If I try to wait for more data than that, the application never returns, the thread (or the app itself) gets blocked!!!!!!!!!!!!!!

... and I'm not using them anymore, that' not the problem!!!!!!!

Thanks,

JaaC

If you can't get it working without threads, I doubt that you'll be able to get it working WITH threads.

Are you certain that the transmitter is still trying to send data at that point? Are you monitoring the progress in the transmitting end to be sure that it is not at fault? How does it respond if the PC is unable to accept data for some interval?

Mark Borgerson

Hello,

You were right, Mark.

It was my mistake, a quite stupid mistake from my part, I must admit it. At that point the DSP stopped writting data to the USB FIFO. Let me explain why, to share the experience, as I promised:

I use the USB FIFO to write data from the DSP to the PC in the most independet possible way. I mean, I don't want much intrussion from the trasferrring process into the DSP-tasks performed by the DSP, so to speak. In that order of ideas, I use DMA to transfer the data from the DSP to the PC. This is the scenario: every 26 mSecs, I set up a DMA process, and fire it away, so the DSP can continue doing number crunching while the on-chip I/O processor sends the data to the PC, thru the USB FIFO. There is an acknowledge signal, which I connect to the TXE signal in the FIFO for "handshaking". I also set up some waitstates in order to meet the timing requeriments of the FIFO, which is somewhat slower than the DSP.

For the DMA process you must set up basically these parameters:

1) Internal memory

- Base address of internal buffer (where the internal DMA data buffer starts)

- Length of internal buffer (how long is it the buffer)

- Stride (the "step" at which the internal DMA pointer increments)

2) External memory

- Base address of external buffer (keep in mind that the FIFO is a memory-mapped peripheral, so this is the address of the FIFO)

- Length of external buffer (how long is it the buffer)

- Stride (the "step" at which the external DMA pointer increments)

My mistake was that I had a non-zero external stride AND I WAS NOT UPDATING THE BASE ADDRESS OF EXTERNAL BUFFER, so, after a few (262144, to be precise) writes to the FIFO, the address at which it was attached was not being decoded, so I was not writting the data to the USB FIFO, as you (Mark) suggested!!!!! The solution was to update the address of external buffer with every DMA.

Other solutions could have been:

1) Lengh of external buffer = 1 2) External stride = 0

For the PC side, I am using this approach (notice the much bigger value of 1500000 bytes I used for testing, and worked):

Actually, what the 2nd thread does is to check how much data have I received from the USB FIFO, and stop when I have collected enough data.

void CTestMThreadDlg::OnReadUsb() { // Create thread for data reading CWinThread* pCommsThread = AfxBeginThread(USBCommThread, this, THREAD_PRIORITY_TIME_CRITICAL); // Create thead for data parsing CWinThread* pParseThread = AfxBeginThread(ParseThread, this); }

UINT CTestMThreadDlg::USBCommThread(LPVOID pParam) { CTestMThreadDlg* pTestMThread = (CTestMThreadDlg*) pParam;

FT_STATUS ftStatus; DWORD eventDWord; DWORD rxBytes; DWORD txBytes; DWORD bytesReceived; char rxBuffer[65536];

HANDLE hEvent = CreateEvent( NULL, false, // auto-reset event true, // signalled state ""); // Check the value of the hEvent!!! if (hEvent == NULL) { DWORD lastError = GetLastError(); AfxMessageBox("Error %d when tried to create handle\n", lastError); return -1; } else { DWORD eventMask = FT_EVENT_RXCHAR; ftStatus = FT_SetEventNotification(eventMask, hEvent);

if (ftStatus == FT_OK) { gettingData = true; while (gettingData) { WaitForSingleObject(hEvent, INFINITE);

ftStatus = FT_GetStatus(&rxBytes, &txBytes, &eventDWord); if (ftStatus == FT_OK && (eventDWord & FT_EVENT_RXCHAR) && rxBytes

ftStatus = FT_Read(rxBuffer, rxBytes, &bytesReceived); circBuffer.Write(rxBuffer, bytesReceived); dataReceived += bytesReceived; } } AfxMessageBox("USB Comm thread terminated"); return 0; } else { AfxMessageBox("Error setting notification\n"); return -1; } } }

UINT CTestMThreadDlg::ParseThread(LPVOID pParam) { while (dataReceived < 1500000) { } CString str; str.Format("%d byes on memory!", dataReceived); AfxMessageBox(str); gettingData = false; dataReceived = 0;

AfxMessageBox("Parse thread terminated"); return 0; }

I've found the two-thread scheme particularly useful because that way the main application doesn't get blocked by the USB-data-waiting process, and I can stop it if I want.

Of course, it still needs some polishing, but at least I have the USB side of the application done!!!

I want to thank all of you guys who helped me to find a solution for this problem, thank you _for your patience_ and your support.

And yes, FTDI parts are great for USB projects!!!!!

Regards,

JaaC

Glad to see you figured it out. Now that you've got it working with threads on the PC side, I'll keep a copy of that code around in case I ever get back to a PC application that needs threads.

Mark Borgerson