StrongARM-based Intrinsyc Cerfboard and Compact Flash craziness

Hello, We have been working a project that utilizes a StrongARM-based Cerfboard from Intrinsyc (SA1110 version of the StrongARM). For those of you not familiar with it, the Cerfboard is a small embedded module that runs Linux (specifically, a distro called I-Linux that Intrinsyc provides). This board has Compact Flash, 10/100 ethernet, and a few other goodies. Anyway...

Recently, we have started using the Compact Flash slot that is available on the board. We noticed that at times, the board would completely freeze when attempting to go through the cardctl insert/mount procedure to gain access to the Compact Flash. We wrote a script that would do the following:

Forever cardctl insert mount /mnt/cf cp file to CF rm file from CF umount /mnt/cf cardctl eject print iteration count

We then let this run forever, watching the output on our serial console. Sometimes, the board will crash after anything between 0 and

40 iterations (random). There is no response from any i/o device (serial console, ethernet, etc). However, upon scoping the SDRAM lines, there is still activity here, albeit much less than before the crash occurs (by about an order of magnitude).

Sometimes, the board runs forever just fine...

...HERE IS WHERE THE CRAZINESS PART STARTS

One of our technicians noticed that if the script is started in the FIRST nine minutes after power-up, the script will fail anywhere between 0 and 40 iterations. If we start the script AFTER nine minutes (its not PRECISELY nine minutes, but close), it NEVER EVER FAILS! We have had it run thousands of times without any problems, as long as we wait nine minutes before starting.

Here are the specifics of our setup:

Kernel: 2.4.9-ac10-rmk2-np1-cerf2 Card Services: 3.1.22 CPU: SA1110 rev 8 Distro: I-Linux 4.4 Execution speed: 192 MHz (through PLL)

There are no applications running on this board, other than the script.

From a hardware standpoint, we do notice that after the nine minute

mark the address/data bus in general appear much less active. Seems as though prior to nine minutes, there are address/data accesses on the SA1110 busses about once per 1.5 uS or so, in burst-chunks of about 14 at a time, with about 6 uS between burst chunks. After the nine minute mark, the burst pace is about once every 14 uS, still in groups of 14 at a time, with a time-betwee-burst chunk also increased linearly.

Has anyone ever seen anything like this? Any ideas what magic happens at the nine minute mark? We have been eliminating hardware slowly, and are starting to think it is kernel/driver related. As a test, we grabbed an XScale-based Cerfboard running 2.4.18 kernel, and same Card Services. We ran the script on this board and it doesn't exhibit the problem (though the hardware is obviously also different).

If you've read this far, much appreciated. We've been pulling our hair out regarding this one. I don't think Intrinsyic still supports this board, but we're following up with them just in case.

Thanks in advance for any help that can be provided.

Balding, John O.

Reply to
john.orlando
Loading thread data ...

We worked on a CerfBoard with SA1110 ;-) You have to switch to a more recent 2.4 kernel, I suggest you at least linux-2.4.27.

Best regards

--
Marco Cavallini
=====================================================
 Click to see the full signature
Reply to
Marco Cavallini [KOAN]

Hi Marco, Thanks for the info...the 2.4.9 kernel was the last one that was "officially" provided by Intrinsyc, which is why we were trying to stick with it. We did get a 2.4.18-based kernel up and running on our SA1110 board yesterday after tweaking a few things. However, I don't believe that this fixed our issues. We can try to get to the 2.4.27 kernel...out of curiousity, how did you pick this kernel, other than the fact that it is near the end of the 2.4.x series? Is there a specific fix in 2.4.27 that you are aware of that fixed an issue you had?

Also, out of curiousity, since Intrinsyc apparently doesn't support anything "officially" past 2.4.18 (and even that is only for XScale-based boards), how did you go about building and patching your kernel to get it to work? Did you just go to

formatting link
and grab the appropriate kernel and patch it against Intrinsyc's updates?

Any help is appreciated here...thanks!

John O.

Reply to
john.orlando

Which rmk patch are you using ?

I used 2.4.18 version years ago, and it fixed some issues, you can see release notes here

formatting link
I sent a patch in linux-2.4.27-vrs1 but is for (Cypress USB host)

If I remember well I solved CF problems upgrading Card Services.

Get vanilla kernel-2.4.27 and the patch linux-2.4.27-vrs1 from

formatting link

BTW The most important thing is to switch to latest Card Services: 3.2.8

HTH

--
Marco Cavallini
=====================================================
 Click to see the full signature
Reply to
Marco Cavallini [KOAN]

Kernel: 2.4.9-ac10-rmk2-np1-cerf2

We did this...so now we have a 2.4.27 kernel with the vrs1 patch in it. We don't have the ethernet fully up and running yet, but the kernel boots and brings us to a login. Alas, the problem still exists. Problematic in the first nine minutes, perfect after that.

We upgraded our card services to 3.2.8, and this didn't seem to make any difference.

So...we are still scratching our (almost bald) heads at this point. Any other ideas out there regarding what could possible be going on here?

Thanks, John

Reply to
john.orlando

Where can I get this reference board?. Also does they provide all necessary Linux OS, also I need Ethernet port on the ref. board. Thanks.

Reply to
santa19992000

Well...it looks like we solved our problem. If you haven't followed the thread up till now, I'd suggest re-reading it from the beginning. I'm just re-capping the solution we found.

We kept trying to figure out what could be causing a change in the system at the 9 minute mark, and why we could use the Compact Flash in our StrongArm-based system without problems after the 9-minute mark, but with tons of problems before the 9-minute mark. We decided that perhaps we could get more info regarding the current state of the system if we could take a snapshot of all the registers in the system once a minute, and see if anything changes around 9 minutes. There is a module called registers.o that can be insmodd'ed which allows all the registers in the SA1110 to appear as regular files. So, we wrote a script to go through and once a minute dump the name of each register and its contents to a file. We let the script run for 30 minutes, and then examined the output log. Sure enough, at the 9-minute mark, there were a few registers that went through a one-time modification, and a few registers that had been changing every minute up till the 9-minute mark, and then stopped changing. We started digging through the SA1110 data sheet to see which registers these were. It turned out the majority of them had to do with a DMA channel that is allocated to support the framebuffer for an external LCD display. We aren't using any LCD display in our system, nor do we need the framebuffer. This support was turned on in the stock config file provided by Intrinsyc for the CerfBoard. So, we re-built the kernel without framebuffer support, and sure enough, our Compact Flash test didn't fail regardless of whether or not we started it in the first 9 minutes. We re-ran our log-the-registers-to-files script as well, and the registers that had previously been changing at the 9-minute mark were no longer changing. So, we are guessing that there is something screwy with the framebuffer/LCD driver that was somehow causing a conflict with our Compact Flash card. The 9-minute thing is still a mystery to me though. But as I said, we don't need framebuffer/LCD support, so out of the kernel it goes.

We will now start the long, slow process of regrowing our hair :-)

Thanks for all who offered an idea...as always, much appreciated.

John O. snipped-for-privacy@gmail.com wrote:

Reply to
john.orlando

ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.