Technical vote hacking most likely did NOT occur in the 2000, 2002, 2004, or 2006 elections.

I recently watched the HBO documentary "Hacking Democracy". Like all documentaries, it has a specific point of view brought to it by the producers that is not entirely accurate or complete.

As a result, certain persons and groups interviewed by its producers have been, in my opinion, treated harshly or unfairly, including the Volusia and Brevard County Florida Supervisors of Elections, and CIBER, Inc of Huntsville Alabama.

I would like to offer my own take here on Hacking Democracy as an experienced software engineer and Democratic party activist, and, hopefully help put "Hacking Democracy" into a better context.

Contrary to what the documentary implies: Technical vote hacking most likely did NOT occur in the 2000, 2002, 2004, or 2006 elections. Please note that I wrote technical fraud; there were other dirty tricks that the Republicans engaged in which I'll address later.

But first, let me share with you some of my qualifications to discuss the issue of software and the vote-counting process.

I am an automated test equipment (ATE) engineer. I write software that performs data acquisition, hardware control, communications/networking and hardware testing. I have been doing this sort of work since 1996 for a succession of companies around the country as a contracting engineer. I have also worked as a software test engineer whose purpose was to test the behavior of software against specifications for FDA safety testing of hemodialysis machines.

In many aspects these tasks are very similar to the tasks performed by automated voting equipment such as that produced by DieBold, ESS, and others.

I also have a substantial background in campaigns and voter database issues:

I provided substantial volunteer I/T support to the Orange County Democratic Party and to a variety of Democratic candidates for office in the 2004 and 2006 elections including the Kerry campaign, the Rod Smith campaign for Governor, and several other candidates. Also. I am also a precinct committeeman for the Democratic Party here in Orange County Florida where I live. (Paul -V-'s Note: An added qualification that Doug did not mention is that he wrote the definitive handbook on IT security for activists.)

I will now address the primary problems I observed in HBO's documentary Hacking Democracy.

I) The constant and ominous references to negative ballots and counting backwards have a simple, more innocent, explanation.

The counting backwards would most likely occur at 32767 votes because numbers are probably being represented in the computer program as 16 bit signed integers. After 32767 votes added would "roll over" the count and cause it to start going backwards. (For the non techie people out there, think of this as being like your car odometer rolling over at 99,999 miles.)

This problem was almost certainly not malicious on the part of Diebold, ESS, or other vendors but rather a simple coding oversight by the programmers of the software who did not consider the unintended consequences of choosing signed variables or short (16 bit variables) with respect to the roll over issue and the possibility of pre-biasing the votes on the cards.

Variables in voting machine software should never be declared as "short"; instead they should be declared as "unsigned long": which means unsigned 32 bit integers, or in other words, that negative numbers would not be possible and that the count would go between 0 and 4,294,967,295 or slightly over 4 billion.

This exceeds the entire population of the United States by a factor of 14 (whereas 32767 is small enough to actually occur in real races) and would prevent the roll over scenario from ever occurring and it would also prevent malicious persons from preloading the vote counts with negative numbers as demonstrated in the video by BBV's security expert.

In other words, the backward counting was not a grand conspiracy on the part of Diebold. Their big sin was not being upfront with a system that appeared to have had multiple technical defects and how they planned to correct them.

II) I would not describe the simple manipulation of voter database tables stored on SOE Sanchez' computer system as a "hack". That really stretches the term "hack".

All that was done was the opening and manipulation of a standard database file (it appeared to simply be a Microsoft Access .mdb file) using some simple SQL queries which did the obvious and expected thing - it altered the stored records. It wasn't really much of an impressive "magic trick" to me as a programmer.

Not mentioned in the documentary is that every file stored on your computer has a time and date stamp and a file size and that every NTFS formatted hard drive has a master file table (MFT) and journal logs that track file changes on the hard drive. A simple SQL record change as demonstrated in the video would be fairly easy to detect using common forensic techniques so I would not be willing to describe the vote change demonstrated on SOE Sanchez' computer using the SQL query as "untraceable".

The real issue that was not addressed by BlackBoxVoting.org here was the fact that they were storing voter information in aggregate form as the official election record in the first place.

Regarding the change performed to the database file at the SOE office, I would like to recommend the following safeguards be implemented by SOEs to minimize the risk of hacking in the future:

After an election, individual voting machine data-cards should be immediately backed up to a write once media like CD-ROM or DVD-ROM as they are uploaded so that a permanent record exists that cannot later be easily altered or better yet replaced entirely by write once media like DVD-ROM or CD-ROM so that they can not be altered after the fact. Such write once media must be treated like evidence in a criminal case with a chain of custody, seals, and secure storage.
Voter data should NOT be stored in relational databases as simple aggregate totals. This makes it too easy to alter the results en masse. It should only be presented that way for final viewing on the SOE website.
Voter data should be stored in a journal or transaction log format where each individual vote is individually recorded into a file along with other corroborating information like:

Software Version ID and an MD5 (CRC32) checksum for that software.
Voting machine serial#
Voting data card serial#
SOE Operator#
A globally unique voter sequence (GUID) for that particular machine generated by a remote SOE machine that provides this to all voting machines. It in turn would keep track of which machines it issued voter sequence numbers to and when so as to make it hard to fake without access to both machines.
Time and date stamp.
Vote totals for each race for the machine as well as how the voter voted in each race.
A CRC32 (cyclic redundancy check) number computed for each transaction. This is a complex checksum algorithm that would make it very difficult if not impossible for a "hacker" to go back after the fact and alter the records and is the basis for file verification when downloading software and also in many common anti-virus programs.

You could also add the actual voter ID number from the SOE rolls to this list of items to insure additional security but then it would be possible to determine how people voted. Tracking the operation of each machine like this in a transaction log format rather than in simple relational database tables with aggregate counts would make it much much harder to fake the results because the votes would then have to be faked one by one rather than en masse as demonstrated in the video. It would be very hard to fake the votes in a convincing manner that could not be discovered by poll watchers who keep track of when people come and go and how many show up at the polls at a given time. This transaction log, any databases, the hard drives, the data cards and the operating systems must be:

Password protected
Encrypted
Protected by physical security and chain of custody must be insured.
Norton "Ghost" imaged or otherwise forensically copied to a permanent write once media for preservation before and after the election.

It seems like all of these very obvious security techniques were being ignored or by-passed in the Sanchez' demo and I hope that they are actually being practiced in the real world.

III) Specifications testing hardly ever provide a complete check of the functionality of any complex piece of code and should be viewed as a mere starting point in software quality, not an end.

When the government buys custom software for DOD, FAA, NASA, FDA, or other government agencies such as voting software obviously is, the software is not "open source" to the general public - however, the government as the customer is always given the source code for maintenance, inspection and test purposes as well as compiling instructions to allow the government to verify that the delivered compiled code is actually the same as the source code provided, along with MD5/CRC32 checksums to verify the compiled code, design documentation and specifications for testing.

Because they apparently did not understand this simple fact about software testing, I believe that the BBV people treated the CIBER people in Huntsville Alabama rather unfairly in their "ambush" interview. The CIBER people were simply given a check list of test specifications by the purchasing governments which likely as not wasn't complete due to a lack of technical sophistication on the part of the purchasing SOEs and a lack of insight into the development process of the vendors.

The CIBER people simply ran the check list they were given and had no way to test the software beyond that without having design documentation and source code which they were likely not given by the vendor. They did the job they were contracted to do, nothing more, nothing less and it really wasn't fair to bash them for not having access to information that the vendor did not provide to them.

The software is, after all, tested "to specifications" and not "to perfection" which is of course much harder to define. Those specifications, like it or not, are almost always incomplete or faulty in some manner in any event.

Beyond specification testing, an actual source code inspection and analysis by a number of different experts is also a required (although not necessarily sufficient) condition of software quality assurance.

In the end there is never any total assurance that software is entirely bug free and we have seen a number of spectacular examples of software quality failures in recent years including cases like the Mars probe that cratered in a few years ago due to inconsistent engineering units in the software, a lost earth orbiting satellite due to an incorrect constant for the rotational velocity of the earth, a radiation device for treating cancer that delivered lethal radiation doses and others where extensive testing failed to reveal unintentional flaws in the software prior to operational use.

Such mission-critical or safety of "life and limb" type software (sometimes jokingly referred to as controlling "elevators", "nuclear power plants", "airliners", and "iron lungs") is notoriously difficult to make bug free and I have personally been involved in writing and testing such software for wind tunnel systems where control of a 5,000 kW (6700 horsepower) electric motor was involved. It is no joke if you lose control of a 5,000 kW motor - people can be killed and millions of dollars of damage can occur - so it takes a great deal of effort to get it right.

Yet, nevertheless, we continue to have spectacular failures of mission-critical software in a variety of areas.

Even if you dissagree with the three assessments above, I have one important question to ask:

The Republicans knew they were going to lose badly at least 2 months in advance. What happened? Did all the vote stealers go on vacation this time?

Why didn't the GOP put the fix in? Even if they did not feel they could outright steal the entire election, why didn't they pick a few of the closer races, at least in the Senate, to steal? There were clearly several very close races in Montana and Virginia after all where if they were going to steal the election they could have tinkered with these races in very marginal ways to win them.

Based on this election, I don't believe the vote was actually stolen in 2000 or 2004 using technical hacking means, but rather, through the Republican party simply doing a better job of mechanically turning out their base voters than the Democratic Party in combination with old fashioned voter suppression through fear tactics (in some instances) like the ever popular "if you vote you're going to jail" leaflet, campaign dirty tricks in some instances like phone bank jamming, and through a faulty "felon" purge that occurred here in 2000 that was heavily weighted to punish Democrats.

In reality, plain old fashioned gerrymandering of election districts has had the most impact on elections by far here in Florida making most districts uncompetitive simply by very careful selection of election district lines using FREDS data and Census Bureau TIGER line data and other demographics database info to push minority voters into bizarrely shaped super-majority districts such as Corrine Brown's here in Central Florida where she is assured a win every year because of overwhelming African American support in her own district while surrounding districts have little if any African American votes in them causing them to tip to the Republicans since the African American vote is heavily (90%+) Democratic.

If you want to say "the fix is in", then in reality, it is much more about the gerrymandering that occurred in the 2000 reapportionment - at least here in Florida - than about technical vote hacking.

Final conclusions regarding the HBO documentary "Hacking Democracy" and voting methods in general.

1) Technical vote hacking most likely did NOT occur in the 2000, 2002, 2004, or 2006 elections.

2) There are numerous actions that I have outlined that can be taken to significantly harden the voting software against hacking.

3) No software is "hack proof" or "bug free" however so paper based "optiscan" type ballots should be used and touchscreens should be removed from use and a 3% sample hand count of these paper ballots should always be performed as a safety check.

4) The following voting issues which are more serious than vote hacking need to be addressed expeditiously:

Gerrymandering. Blatant district gerrymandering using FREDS, US Census Bureau TIGER line data, and other demographic resources must be stopped and districts need to make organic sense following geographic boundaries like rivers, highways, city limits, etc. so that communities of interest are preserved and so that districts are not biased by racial or economic factors to prevent competitive races.
"Felon" and other types of "purges" whereby voters are removed from the ballot and either denied a vote or forced to vote provisionally when these purges are often subject to cases of mistaken identity given the issue of common names.
Intentional voter suppression "dirty tricks" like the "if you vote you're going to jail" leaflet that appears in election after election in African American and Hispanic communities.
Dirty tricks like campaign phone bank jamming, false messages using robodialer systems, and other such tactics.
Legislation at the state and Federal level needs to be passed to insure that the fixes I have recommended here and other fixes that other experts recommend are implemented promptly to minimize the risk of a hack in future elections.

Read counterpoint here.

Tags: HBO - Hacking Democracy - BBV - BlackBoxVoting.org - Diebold - elections - documentary

Notes: 11/16/2006 @ 14:00 PM EST - This post was edited by Paul -V- and is still being tweaked for spelling, grammar and context. When complete, I'll say so in this footnote.

17:00 PM EST - Editing complete.