Machine Language: What's Your Sign?

Jim Butterfield
Toronto, Canada

Beginning programmers learn very quickly that a memory location has eight bits capacity, so that it may hold a number from zero to 255, or 00 to FF in hexadecimal. That's the range of values that you are allowed to POKE and that you will see with a PEEK. These numbers all seem to be positive. Why, then, do some of them set the N (negative) flag when loaded? More generally — how do you handle signed numbers?

As Humpty Dumpty almost said: "When I use a number, it means just what I choose it to mean — neither more nor less". As programmers, we can choose to treat a memory location value as if it were unsigned, a number from 0 to 255; or signed, a number from -128 to + 127. We can view the value in many other ways, too: as part of a bigger number, as an ASCII character, as two or more small numbers packed together, and so on. For the moment, let's concentrate on the signed number aspect.

Sign Posts

The N (negative) flag is tied to the most significant bit of the value under consideration. For example, if we load into a register a value of hexadecimal C8 which would PEEK as 200 decimal, the N flag is turned on. Why? If we write C8 in binary we get 11001000. The first bit (called the high-order bit or the most significant bit) is one, and that's what kicks the N flag on. If we mean the number to be unsigned, we may ignore the N flag; but if we mean the number to be signed, the N flag tells us that it is negative. We can tell what negative value is represented in an 8-bit location by subtracting its unsigned value from 256, so that C8 (unsigned 200) has a signed value of-(256-200) or -56.

This method of representing signed numbers is called twos-complement and it works well once you get used to it. There are a few special rules to keep in mind when you add, subtract, multiply, and compare, but most things are quite straight-forward. You'll quickly learn that FF or decimal 255 has a signed value of -1; that the highest 8 bit signed value is hex 7F or +127 and the lowest is hex 80, or -128.

Overflow: My Byte Runneth Over...

When we add unsigned numbers, we need to watch for a leftover Carry (C flag) after the addition is complete. If the C flag is on, it means that the addition has generated a result that is too large to fit the space available. Similarly, when we subtract unsigned numbers, we look for the inverse: the C flag being off means that we have tried to subtract a bigger value from a smaller one — and that's illegal if we want unsigned results.

The rules are different when we add and subtract signed numbers. The problem we must look for here is a "sign switch": for example, adding + 100 to + 100, two positive numbers, will generate a value of 200 — which is a negative value if placed in a single byte. This type of error is called "overflow," and the 6502 conveniently provides us with an overflow indication (the V flag) to warn us of difficulty in signed addition or subtraction. A BVS (Branch Overflow Set) will detect the fault and allow us to code an appropriate error or warning routine.

Remember that both Carry and Overflow are set with each Add and Subtract command you execute. It's up to you to choose which flags are important: you know which numbers are signed and which are not.

Multiplication: Sign Of The Times?

General multiplication of signed numbers calls for careful testing of both signs and quite a bit of work. For the moment, we'll concentrate on simple multiplication routines: multiplying by a fixed value of say four or five.

We multiply a number by two by using an ASL command. If we were doubling an unsigned number, we once again test the C flag to make sure that the new value fits into the space provided. For signed numbers, it's a little more work: we must make sure that doubling the number hasn't caused the sign to change. The overflow flag won't help us here (I wish it did) since it is unaffected by Rotates and Shifts. The usual coding method is to check that the C flag, which holds the previous sign, matches the N flag, which holds the current sign.

To multiply by four, we use two ASL commands, and we must carefully check for errors after each one. If we wish to multiply by five, we multiply by four and then add the original value — hopefully stored somewhere — and make the final overflow check on the addition.

Comparison: Getting the High Sign...

Comparing signed numbers can't be done with a single flag or a single test. The C flag gives you a valid comparison if the two signs are the same, but not if they differ. You could pre-check the signs: for example LDA VALUE 1 : EOR VALUE2 would turn the N flag off if the signs were the same, or on if they were different. If the signs were identical, a normal Compare would settle the matter; if not, the positive number must be the biggest one.

Frank Covitz offers the following elegant signed compare:

TEST1: SEC
       LDA N1      ;Get 1st number
       SBC N2      ;Compare vs. 2nd number
       BVC TEST2   ;Branch on overflow clear
       BIT N1      ;Else test sign of 1st number
TEST2: BPL GTE     ;If plus, branch to Greater Than or Equal
       ..          (code for less than)
       ..
GTE:   ..          (code for greater than or equal)

Note that two tests are performed: the SBC (which is used to set flags rather than calculate results) and the BIT test. The N flag is brought into play here — it's unusual to see it doing a useful job in a comparison situation.

Odd Signs

When signs are used as part of large numbers, the sign bit appears only in the most significant byte. So if you allocate 32 bits (four bytes) to a value, only one — the highest order — gives the sign.

It's possible to sign decimal numbers. If the numbers are held in BCD for decimal addition and subtraction, the sign works out rather oddly. The high bit is still the most convenient to use — but this causes positive values to be those starting with the digits 0 through 7, and negative values to begin with digits 8 and 9 only. This "unbalanced" arrangement of numbers is often satisfactory and allows the N and V flags to perform their proper roles. If, on the other hand, you need to balance the range of positive and negative decimal values, you'll want positive numbers to start with digits 0 through 4 and negative values with 5 through 9. In this case, you have to do most of your own sign work. As a last resort, you can keep the sign as a completely separate flag — but beware of additions and subtractions that cross the positive-negative boundary.

Signing Off...

Most machine language work is in unsigned integers: you'll need to deal with signed numbers only rarely. But when you do, it's essential to know how to handle them ... you might say that it's one of the signs of good programming.