Here is some new bad news, and i mean really bad news

Of course it does. The coder used an autoincrement pointer to pick up a buffer size and dumped that amount of memory, without bounds checks, addressing memory whose content was entirely unknown. No programming language should allow that.

The cultural part is that buffer overrun errors have been chronic security hazards for decades, and neither the coder nor the open-source peer reviewers checked for it! Some unchecked buffer bugs can be very subtle and hard to analyze, but this one was blatant, simple and in plain sight.

If strong checking eliminated half of the security bugs, that would be a big improvement. In most engineering situations, 2:1 is considered to be a big deal.

Buffer overflows are common. If you're lucky, they crash the OS and you find out and fix it. If you're not lucky, some bad-hatter finds them before you do.

The worst thing that technology does is program. Critical systems are programmed in a 45-year old, fundamentally hazardous language that requires execptional, K&R-level, skills to get right. Will we be doing it the same way, in c, 45 years from now?

c is fine for small bare-metal uP apps, where a single programmer does the entire app, and the consequences of a bug are small. In bigger contexts, you get heartbleed, and literally thousands of similar hazards.

So, are we in for another 50 years of the same? You seem to be arguing "yes."

What are your favourite static analysis tools, Martin? I mostly use PCLint, but I don't have an uplevel copy. I'm a big fan of mudflap for debugging.

Cheers

Phil Hobbs

PowerBasic doesn't put a limit on string length, allows embedded nulls, and has groovy inherent string functions. Without hazards. Ask for a substring out of the range of a string and you get the null string. Append to a string and it just works.

I've written PB programs that manipulate huge data arrays, using subscripts, that run 4x as fast as the obvious c pointer equivalents. With an afternoon of playing with code and compiler optimizations, the c got close.

Actually he didn't that part was hidden inside the memcpy routine.

The problem was not sanity checking the parameters in the message for validity. This is all too common :(

It was only in plain sight if you think like a black hat. You can hide a lot of things in plain sight. Most people don't consider what would happen if someone were to deliberately seek to find hostile parameters.

Strong typing would catch some of the faults. *Using* the available static analysis tools for C/C++ would also improve things, but the most important thing by far would be making sure that the next generation are taught from the outset to use these tools.

It is unfortunately the case that the best tools are only available to a select few with very deep pockets or on a roll your own basis.

They are not as common as they used to be. Steve McGuire (sp?) in his classic "Writing Solid Code" described how it should be done back in the

Most of the realistic alternatives are of a similar vintage. New silver bullets promise everything but then deliver little or no improvement.

Crucially if you already know how long the heartbeat datagram should be then you should drop it on the floor if it is clearly malformed on arrival. Data corruption can sometimes happen so there is no excuse.

I hate to defend C but in this instance it wasn't so much the language as the failure of the handler algorithm to sanity check the parameters it was passed were correct before copying the data and sending it back.

The problem arose from trusting the value of "payload". Most attacks against computers arise from misdirection or social engineering of the humans using the computer rather that direct technical attacks.

Unless and until someone comes up with a new paradigm that is the difference between machine code and compilers - basically yes :(

You can have formally verified correct software iff you need it but the price is astronomical to do it for any non-trivial application. Formally proved hardware like VIPER has all ended in tears and lawsuits

It will happen eventually that someone devises a graphical programming interface that allows domain experts to codify their knowledge reliably but until then we have to live with the tools that we have available.

In some ways electronics simulators and silicon chip design software are amongst the most powerful tools for avoiding human error we have. They are not perfect but they are a lot better than doing it manually.

I think you will find it limits maximum string lengths at 2^31-1 or possibly 2^32-1. Older basics tend to limit it at 2^16-1 = 65535.

Memory was a rare expensive commodity when these languages were born.

Only because you don't know what you are doing.

Of QuickBasic specifically, it was 32ki, because of architecture (8086, real mode), not because of memory limitations. The string is limited to one segment and has a signed 16 bit integer length parameter, compatible with the INTEGER data type; QB has no unsigned integers.

There were a lot of bad compromises made in the language: having only signed data types, it wouldn't make sense to offer 64k strings when you can't even return the value of a LEN(string$) function call half the time. I wouldn't call that an 'arbitrary limitation', it's partly practical (saves having to specify an unsigned data type) and partly architectural.

I don't remember if the STRING's base address was segment (16 byte) aligned, but allocations of that sort were common. Every STRING was a WORD for length, followed by the value (including NUL, which is just another character). I guess some people don't like the idea, maybe because it makes all the offsets off by 2 (or 4 for a uint32_t*)? An extra ADD or SUB instruction, BFD.

One thing that was perversely valuable: STRINGs are the only dynamic sort-of-object-oriented primitive in QB. So you'd often use strings as dynamic arrays, lists and so on. All horribly inefficient of course (every time you work on a string, it has to be re-allocated and copied), but it was the only way.

As I recall, you can tell QB to allocate much more heap and thus use more than 64k total of strings. That might be wrong. I do recall there's a "long" mode which allows you to allocate and handle arrays over 64k in length (of arbitrary indices, dimension and data type); they warn you it's slower (of course, because of the segment calculations required).

Tim

See e.g.

Cheers

Phil Hobbs

Almost, but not quite, entirely disagree (with apologies to Mr. Adams).

It would have been feasible to make C pointers size-safe. After all, a pointer is just an opaque object of a certain size (typically one machine word) containing certain information (typically a memory address). The C language does not even specify that the above "typically ..." statements must be true. It's entirely feasible to write a C compiler that encodes a pointer as something else entirely, like a structure with additional fields or like an abstract "handle" that is interpreted by some sort of runtime library routine. It's not efficient to do it this way, but CPU cycles are getting cheaper and the penalty is not as bad as in fully managed runtimes.

Sure enough, C has so called "pointer arithmetic" operators like +, -,

The above may look a little too theoretic, so here's an example how a "pointer safe" C compiler might do it (examples in c-like pseudocode):

Assume a data type for a machine word capable of holding a memory address:

typedef int machine_word; // example using an "int"

In reality this would be a CPU-dependent "compiler intrinsic".

Assume a "pointer" data type internally uses the following representation:

typedef struct { unsigned machine_word base; // allocated memory segment unsigned machine_word size; // allocated size of the above unsigned machine_word itmsize; // item size for arrays unsigned machine_word address; // the current location } pointer;

In this case a pointer variable would be 4 machine words long internally. Of course this would also be a compiler intrinsic, the internal layout does not need to be user-accessible. The compiler generates the structure and fills the fields as needed (for exceptions see below).

A pointer declaration like "char t[42];" would have the compiler generate this:

pointer t = { .base = 0x1234ABCD, // real memory address of the buffer .size = 42, // total buffer size (in bytes) .itmsize = 1, // size of basic data type (here "char") .address = 0 // beginning of the buffer. };

Note that this is compiler-generated, the user actually writes "char t[42];".

A dereference operation ("*" in C) would internally work like this:

machine_word dereference(pointer t) { if (((signed) t.address) < 0) throw DEREFERENCE_RANGE_ERROR; if ((t.address + t.itmsize) > t.size) throw DEREFERENCE_RANGE_ERROR; return t.base + t.address; // use address safely in range }

Again, this would be compiler-generated, the example is to show how it works internally.

A "t++" would internally look like this: { t.address += t.itmsize; // dereference will catch overflows later }

A "t--" would internally look like this: { t.address -= t.itmsize; // dereference will catch underflows later }

And so on, ... similarly for the remaining pointer operators.

Loops like "while(d++=s++);" would be safe with this implementation ("++" incrementing the "address" field and "dereference" doing overflow checks). Other pointer arithmetic loops would similarly be intrinsically safe.

If implemented consequently, this would still essentially be compatible with the specified behaviour of the "C" language and the programmer would not even need to know the compiler details (he'll only notice that pointer variables take 4 times more space than usual and pointer arithmetic loops run a little slower).

Of course the world is not perfect, so there are some tricky bits left.

Consider this:

{ int i = 0x1234ABCD; char *p;

p = (char*)i; // typecast integer to pointer, legal in C }

In order to stay within C semantics, the compiler would need to implement the typecast like this:

{ p.base = 0; // physical address zero p.size = 0xFFFFFFFF; // physical limit p.itmsize = 1; // size of char p.address = i; // unsafe, but no other choice }

This would create an unchecked (impossible to check) pointer with all safety features intentionally turned off. The typecast would work, as per the C standard, but it throws safety out of the window (also as per the C standard). Fortunately the integer-to-pointer typecast is already considered very bad practice and causes decent C compilers to emit a warning anyway. The warnings at least are easy to detect.

A typecast in the other direction would also work and turn "i = (int)p;" into something like

{ i = (int) dereference(p); }

and of course also emit a warning. Not completely safe, but at least detectable through compiler warnings.

Special functions that make memory allocations like "malloc" would also need special implementations (they have to know the underlying structure in order to fill the fields appropriately), but fortunately such functions are few and far between, and they are highly environment-specific anyway, so it would be OK for the implementors to have to use some purpose-built compiler intrinsics in order to construct pointers "from scratch". Normal user code does not need to do this.

Function pointers would need a special encoding too (likely setting .address, .itmsize and .size fileds all to zero), but they are not incrementable anyway.

Compiler-generated wrappers around OS functions imported from DLLs would probably use integer-typecast-like semantics (not safe) but that's unavoidable because safe size and type information for such pointers simply isn't there in the first place. That's a language-independent limitation and it's impossible to fix it inside a language or a compiler directly anyway.

As you see, no solution is foolproof, but a C compiler could still go a long way to improve pointer safety without sacrificing language semantics (at some performance cost).

Regards Dimitrij

Oops, should have been "while(*d++ = *s++);" of course. Too much C can really mess up the "programmer" :)

that is obviously because the database was written in c ;)

-Lasse

;)

Cheers

Phil "C++ fan" Hobbs

That's what you get when you can't tell data from code.

Well, in a 32-bit program, strings won't get much bigger than that.

It's not usually much of a limitation.

Oh, it was a senior c programmer who wrote the c version. It ran slower than I thought reasonable, so I tried it in PowerBasic, which only took a couple of minutes to write and run. It was a fairly simple signal averaging thing, but on big data sets.

functions.

buffer

memory

that.

No. Doofus. The code should have been, by design, send back a single byte. The error was a design error. Nothing to do with autoincrement pointers, bounds checks or anything else. I guess you did not understand the XKCD cartoon very well or ignored it.

security

reviewers

to

It was not a buffer overrun either. It is becoming rather clear that you can't get past your prejudices to look at it rationally.

?-)

code

optimize.

strive

job.

No. No. No. Code reviews properly done in the right spirit (formal systems consistency check) are amazingly cheap. Don't let the bean counters and obstructionists snow you, it is another big lie. It is portrayed as a big expense but, design reviews are the single most effective method (even per dollar) of improving engineering results. Something like 100 studies to

Of course if they are treated like a nuisance to be gotten rid of with the least effort they don't work very well, but all the other methods used in that setting suffer the same fate, gigo.

?-)

I don't see what C's got to do with this. This concerns some form of SQL.

You're supposed to sanitize all input data, no matter what language.

Jeroen Belleman

P.S. Clean up your posts, or better yet: Lose Google.

It is *nothing* to do with that at all. It is a failure to check that the incoming message is correctly formed and then acting on it blindly.

Harvard architecture has made some inroads in embedded PICs. Data and code are in separate memory spaces and completely separate.

Intel provided the means to have separate CODE and DATA but the flat no distinction memory model of the early Motorola CPUs is in vogue now.

OS/2 actually used Intels segmentation to keep code and data apart. It was incredibly robust and if some process failed the rest all kept running. It was fast enough to simulate 16550 UART in software.

Windows flat model won out due to IBMs crap marketing department.

Turbo C didn't have that string lenght limit on that architecture ... you could certainly have a string over 512Ki and possibly over 700Ki if you pulled out all the stops.

c has got known sizes now,

Agreed although it is still subject to human error.

It seems to me that programmers do not think about preconditions, post conditions and invariants that can be used to detect faults earlier.

This latest fault was in effect a failure to check the preconditions for correct operation of the heartbeat code were met and the attack consists of basically violating the rules with a malformed message.

In a sense programmers are too trusting and testers not wild enough.

Murphy's Law applies in spades to software in the field.