On the Bleeding Edge

It’s a decent job—lots of “bleeding-edge” technology and every day is a learning experience. Of course, half the time I would tell you it’s horrible. On balance, since I sometimes refer to the company as “we,” it is apparently a good enough job to seduce my on-and-off allegiance.
I am, in essence, a glorified file clerk: a Database Administrator, or DBA. Actually, my business card calls me a “Database Engineer” but I think that’s either wishful thinking on the part of my employers or one of those “title-instead-of-money” deals. My job is to keep track of a lot of information—no different than any other clerk’s tasks.
Traditional file clerks usually only deal with small amounts of documents—a few hundred thousand, maybe. The Pentagon devised a unit of measurement called a “linear drawer foot”—one foot of closely packed documents—to describe the total capacity of some of their stores of documents, which even in the 1970s were measured in miles. The principles remain the same—be able to find “stuff” quickly.
A vague analogy can be made with pilots on combat missions; long periods of boredom interspersed with moments of terror. OK, my terror is not for my life but the principle remains the same. For example, about once a week I am “on call,” meaning computers that I have never seen send me messages about problems I don’t understand. I have a list of instructions to follow which mostly resolves the issues. Most of the problems can be passed on to someone else—networking issues, for instance. But others become “mine” and we then enter into an intimate relationship, the problem and I. So far I have resolved, or at least explained away, all of these. But one day I may be handed a problem I can’t solve and then the company will replace me with someone who can.
Busy doing what, you might ask ? Or does it really matter? The conditions I am describing exist throughout the industry from Silicon Valley through “Silicon Gulch” (Austin) all the way to the 128 corridor around Boston, and for all I know, all the way to Mumbai. The ‘product’ is of little importance, as long as it makes a profit.
A goodly portion of my work for many years is best summed up as “helping businessmen count money faster and more accurately” (I’ve worked in banks and for VISA, among other esteemed handlers of currency). This is the core of most computer professionals’ jobs, at least in the “applications” world; people who make operating systems and other tools are more akin to workers who make the machine tools that companies like GM use to make cars.
And we do indeed have a “product”—pictures. We sell aerial and satellite imagery both from a web site and on CD/DVD. The company does not produce the images—they are bought or rented from companies that have satellites or fly the aircraft that take the pictures. My job is making sure that when some client (an architect, a district attorney, a city planner, a “hi-tech” worker in India digitizing maps of roads, etc.) looks at a picture of some sagebrush outside of Phoenix, we know how much money we got for it, and that the proper cut goes to the owner of the image (“Royalty Check, honey” in Frank Zappa’s words). At peak we produce about 25 images a second, which can work out to a lot of companies accumulating absurdly small amounts of money (forty percent of one-third of one-half cent). But hey, no amount of money is absurd, it adds up, right?

There are interesting contradictions in this product. We spend more computer time (which may in a sense be equated with money) making a large image than a small one, so the company likes to charge by size. Sensible enough, as far as it goes. But whenever someone looks at our imagery—whether browsing or just window-shopping—we splash a logo over it to make it worthless for resale. In so doing we “burn quite a few cycles,” i.e. spend computer time to add the watermarks. Of course, sometimes there is a charge for nothing at all; we charge extra money to show little lines with text—representing roads, for instance—because it takes additional cycles to figure what roads are in the area, but if you use this feature and draw an image of a place with no roads, you still get charged—knowing that nothing is there is information, too.
I get paid well—about twice the median income of people in the San Francisco area; when I was hired three years ago it was on the upper side of wages for comparable work; with no raises since then my real income has decreased by a measurable percentage. Bonus ? You get to keep working next year (actually, I was given a Christmas bonus for 2003—$100.00!). The dollar amount disguises the long hours—lots of our work needs to be done at night at home because the computers are less busy and we will cause less disruption to paying clients. On the other hand, management never can trust the worker to work, so we all have to spend 25+ hours at our desks just so they can see and feel reassured. People commute from the Central Valley—Modesto and Tracy for instance—and are spending hours driving back and forth when they could be working; a terrible loss to business. It is a rare week that any of us logs less than 50 hours; some tend more towards the 70+ work week. Perhaps not coincidentally, there were major layoffs in spring 2002—one-third of the company. Since I started, no less than half of the jobs have been eliminated with a few new hires in sales.
In the past week, as I write this, my boss has quit. Apparently the thinking (if you can call it that) was that her job would get spread over two other people and there would be no impact on delivery dates or site performance. On her last day there was a clash between her sidekick (the head of operations per se, and a very knowledgeable fellow) and management. Sometime between 10:30 and noon he was removed from email and had his accounts shut off. The rest of us responded by drinking rum for the remainder of the day. It will be interesting to see if management continues its policy of reality denial and fantasy. As least part of their psychosis is the belief that software, and the workforce that produces it, is standardized in the same way automotive parts have been. Interchangeability is not simple in the world of computers, or at least outside of the assembly lines that produce the hardware itself. The creation of programs is much more like the craft industries of the mid-nineteenth century. In the meantime, the rest of us are busy trying to do our jobs as well as covering for others.
My stock in trade, as it were, is not the imagery itself—some 25-30 terabytes1 of highly compressed imagery in all. My interest is information about the images—their spatial coordinates, when they were taken, their origin. The databases contain detailed maps of every block of every road in the United States. I’m responsible for moving the data around, keeping it backed up and making sure it’s available when needed. Clerical work at its finest.

In addition to the administrative chores there is a constant pressure from a source familiar to any reader of Capital—the foremost mechanism by which the industrialists make more money is by renovating their plants, whether by upgrading or by discarding old ones in favor of new ones. And so it is in the computer shop, supposedly so far from the industrial revolution—“silicon” is our avatar, not iron.
And yet, curiously, the machines themselves are sometimes referred to as “iron”—as in “heavy iron,” meaning fast computers. They are called, again an echo of earlier relationships, “servers” and are kept in “cages” (because they are dangerous?) on a “farm”in Silicon Valley. I’ve never been to our cage, but I’ve seen photos. It is a chain-link cage in a large building run by some corporate giant. While we are isolated in cubicles, our machines are kept on racks connected to each other (and us) with cables, “switches” and “routers” (specialized computers that move data)—even the simple drawings of our “architecture” are complex.
But having gotten it to work is not enough, we have to replace various bits and pieces. Because of changes in hardware (out go the leased Sun servers, in come the purchased Dell servers), software (Linux instead of Solaris, mostly) and applications (postGres, an Open Source database, replacing Informix, now owned by IBM; old image servers that depended on expensive licensed “libraries” being replaced by new code written in-house, etc.) we have been spending a lot of time replacing almost every component while it is running. Imagine changing almost everything on your car except the chassis and the license plate while driving down the freeway.
At work we use the analogy of driving down a freeway, almost always in the context of driving by looking only in the rear-view mirror. We are constantly monitoring the site but from a certain distance. Billing issues tend to take a day to be seen, while our computer monitors show nice graphs that are only a few minutes out of date at any given instant. To really see what is happening takes actual people. And when something unexpected (i.e. unpleasant) is happening, four or five or more of us will be communicating by voice, phone, email and instant message, sometimes simultaneously. After a frantic spasm of intensely cooperative work we return to our usual tasks.
The daily work is itself intensely collaborative, yet also curiously alienated. Each of us has a focus; the operations people deal with various aspects of the site as a whole, the content people set up new imagery, programmers work on different aspects of the software, quality assurance tests and retests things. This is not a company in which the bosses or managers don’t have a clue—my boss knew her stuff, and the head of the company, although not primarily a computer geek, certainly knows the remote sensing/GIS (Geographic Information Systems) business well. Ergo, mistakes are hard to cover up. As the DBA I need to “work closely with” (i.e. get ordered around by) virtually everyone in the company, from accounting and sales, programmers and ops people. Even my boss and the CEO occasionally give me direct tasks.
There is the usual grousing about conditions common to most workplaces. Yet there is no feeling of solidarity, even among the people I have the most in common with (shared interest in jazz, or cooking, etc.). There’s a shared inaction based in part in the sense that there’s nothing we can do and in part on a lack of trust. Confronted with the inexorable logic of business and cost containment, the ideology of “professionalism” becomes paralyzing. Professionalism means quite a few things—a vaguely positive attitude is a must, and a positive disdain for direct confrontation is mandatory. We adopt the common face and voice to discuss the “problems”—all of which have been specified before we confront them and as such have already had all possible solutions defined before we even see them.
In one of the odd contradictions of such a “professional” environment, we are treated with a certain degree of respect, but we’re all expendable. Even as we watch one of our people hustled out the door after a summary layoff, the most we might do is have a sotto-voce discussion, usually with a friend of the departed.
My attitude is not the best, and I’ve been officially warned that the only reason I am still employed is because everyone who works with me thinks I do a stellar job. The problem? Apparently an anonymous someone has taken offense at some of my emails or IM sessions—no serious vulgarities but perhaps a mild expletive or two. That’s enough, along with management’s irritation at my continuous asking of the old utilitarian “qui bono?” (whose good—who benefits?) when confronted with stupid decisions. We get more and more of those, as the company is owned by a real estate company whose computer types are particularly clueless—they like to put “MSCE” after their names … bragging about being a Microsoft Certified Engineer!
So people show a certain wariness in endorsing my opinions now, at least in public; it is not unusual for people to support me privately, after the fact. Although not allowed to formally question some business decisions, I can at least greet them with all the warmth that they deserve. Not much of a weapon.
But the battle is not necessarily totally one-sided. A slight plus in our column as workers in the software industry is that the process is not well rationalized—not “Taylorized.” It is very hard to predict how long a given (non-trivial) software project will take even for people who know the tools and problem well. There are no easy methods for determining productivity—counting key strokes works for typists but not for programmers—and because the problems are often ill-defined, we can sometimes get time back from the job, help each other by passing the buck on responsibilities and covering for each other. Such small actions do help build the sense of trust, or at least of common ground, that is a prerequisite for more meaningful solidarity.
We also have a shared interest in reliable tools and processes, and the advent of Open Source software—typically software whose “source-code” (original instructions, as opposed to a “compiled” program) is available to all. There are usually groups of people committed to a given tool who work collaboratively for its improvement, even though they may never meet. Applications that are available include graphics manipulation programs, office tools like spreadsheet and word processor, operating systems and HTML servers such a Linux or Apache, programs for creating maps or plotting spatial data, databases and so on. Because the people who create tools have an inherent interest in them there is little need for an incomplete or flawed version of the software to be released simply to meet a schedule. Problems tend to be well-documented and discussed, as opposed to the corporate model, where issues are often hard to discover because of non-disclosure contracts and company perversity. The programs themselves sometimes lack the bells-and-whistles of commercial products, but because the source code is available it can be extended or modified, and there are many people to help with support issues.
As a programmer I gain a better tool; as a person I am sharing in something that has an end result other than some money. It also helps to undermine the arrogant behemoths such as Microsoft and Oracle. The company gets quality software without having to pay endless license fees. One source of tension though, is that the company is benefiting from other organization paying to develop software (the spatial data tool we use was developed by a Canadian company paid by the Canadian government, which did not want to continue to pay large fees to US companies). Yet my bosses are agonized when faced with the need to spend a small amount of money to improve the tool—some other business might be able to benefit from this money! Amazingly short sighted—spend a few thousand to save a few hundred thousand dollars, and then whine about it.
Recent events give me more of a sense of how my co-workers regard the company. A few months ago we were subjected to a company-wide survey conducted by a consultant using a web site. They claimed that all answers would be confidential, but the way we logged in guaranteed that they could track who had said what. So I suspect that the answers they got were slanted in the company’s favor. On the last possible day I answered most of the questions, mostly honestly, after my then-boss got in my face about her group’s low participation rate.
Afterwards, corporate sent a person from “Human Resources” to explain (away) the results. We were generally in line with the company on most of the survey but had responses in two major areas wildly lower than the company averages: benefits and company support for us. Now, keep in mind that the parent company is in the real-estate business, which has a peculiarly exploitative relationship with its workers—real estate agents, for instance, typically get only a commission and then have to pay money to “their”office to rent a desk, etc.
In the session I was in, everyone criticized the benefits. Sales, engineering and operations all criticized the insurance as expensive, “substandard” (this from someone who knows the insurance industry) and difficult to use. Everyone had harsh words for the “401K” plan: 6% is not “matching” the employees’ contributions, and their proposed scheme actually seemed to ignore federal law about limits on employee contributions. Everyone had critical words for our time-off policy as well, again ranging from “illegal” (they don’t roll unused vacation time over to the new year, nor do they pay you for it) to “cheap” and “outrageous.”
The company’s pretty words don’t ever seem to have any money behind them. Fellow employees were not delighted with their pay, either, as most have had no raises for years. On paper the management supports employee’s education, but in practice they have no money for technical classes of the sort I might need (typically one week with about 40 hours of instruction, costing between two and five thousand dollars, depending). We actually got this worthy functionary to laugh when, in the course of discussing how the company does not give us adequate support, we told her that our high-tech company gets hand-me-downs from a local (bankrupt) school system.
I am sure that in subsequent surveys we will simply be asked if we have been adequately informed about our crappy benefits, rather than the more risky ground exposed by the open-ended questions. And because the company is actually making money now on a month-by-month basis, they may actually provide us more of the tools we need to make them more money.
In the short run, however, we’ve had a Company Meeting in which they tried a smoke & mirrors production to pump us up—poorly mixed and stale rock tunes played over a slide show of company content and tools. This was followed with a passionate speech by the president about how hard he had fought for us, the ungrateful employees, when the company was sold to the tejanos. He pointed out that he had no stock or other vesting in the parent company, and was an employee just like us.
This may be true, as far as it goes, but management still is in denial: he was frustrated that only thirty percent of our projects were delivered on time. Given the sparse resources and constantly shifting requirements, doing a third of our deliveries on time is an excellent statistic. According to them, the problem is “communication” so now we’ll spend more time in meetings. As one engineer said to me, “I spend 7 hours in one day now on meetings—how long until they realize that that is seven hours that I am not working?”
We have been put on committees with no power that will be able to make recommendations that management will be free to ignore; or if they are implemented it will be “at manager’s discretion,” a nice way of saying “never.”
It is possible that we can gain some leverage over the situation now. It is clear that there is widespread dissatisfaction, but what exactly can be done is not clear. Hopes of controlling our local bosses are a bit thin; bringing our Texican masters to heel is a rather remote possibility. I can’t see us actually having a picket line, but I think some combination of working only forty hours a week, declining those extra work shifts, and perhaps proposing that we all take time off together might provide leverage. Or perhaps not—there are no guarantees.
Well, it’s 1:30 in the morning, and I have puzzles to solve before I sleep.

the digital salute

Dissatisfaction tends to make itself known, although sometimes in ways that are hard to see. For example, one company that makes digital maps of streets found a curious set of lines in some work. The regular QA people had found no problems, but there was an automated QA process that examined all of the incoming work, and it applied rules that would be impossible for a human: in a computer model of roads there will never be a road segment that is not attached to other segments. Yet in this particular batch they found a number of lines attached to nothing else. When they zoomed all the way in they could see these lines with no labels or other data, but they made no sense. When they zoomed out to look at the whole US the lines couldn’t be seen because of the way scaling and zooming work. Eventually they wrote a special filter to show just the lines with no connections. It made a large sketch of a big “fuck you” with an upraised finger in salute. Alas, these lines were removed before the world at large ever saw them, but it makes you wonder what else might be out there.

• 1 a terabyte is 1000 gigabytes, a gigabyte is thousand megabytes, and megabyte is a million characters, if you care