Posts about transparency

Big Brother’s Big Brother

Here are paragraphs about Wikileaks I just inserted into the chapter of Public Parts that I happen to be writing right now about government. Very much beta. Take a look:

* * *

Wikileaks has pushed the definition and question of transparency to its limit and beyond, releasing hundreds of thousands of leaked documents about the wars in Afghanistan and Iraq through media organizations including the Guardian, The New York Times, Der Spiegel, Le Monde, and OWNI, a French site devoted to digital journalism that built a crowdsourcing tool so readers could cull through the docs to find important bits. The U.S. government screeched indignantly about the leaks, calling them illegal and dangerous. But then, the leaks revealed government actions that are or should be illegal. Who holds the higher ground?

The media organizations Wikileaks worked through said they redacted names and published only documents that would not endanger individuals. So they decided, in the end, what would be secret. Whom do we trust more to make that declaration: government, the leaker, Wikileaks, or the press? And does it much matter now that any whistleblower has the power to leak information anonymously via computers that run in countries beyond the arm of the law from other countries? Wikileaks’ Twitter profile lists its location as “everywhere.” Now nothing, not even war, can be carried out in assured secrecy.

The only solution to leaks is then not more secrecy but more transparency. If we trusted government to determine what needed to be secret—if its default were public and it had nothing else to hide but things that would be harmful if public—then leaks would be a clear violation of our norms and of the common good.

One way or another—by force of through sanity—we are at the dawn of the transparent age. But it’s not going to be a pretty or easy transition. For the first facts to be dragged into the sunlight will be the ugly ones that somebody thinks need to be exposed. Only when and if government realizes that its best defense is openness will we see transparency as a good in itself and not just a weapon to expose the bad. Only when governments realize that their citizens can now watch them—better than they can watch their citizens, we hope—will we see transparency bring deterrence to bad actors and bad acts. Then we become Big Brother’s Big Brother. Or we can hope.

A taxonomy of transparency

I’m writing the section of Public Parts on truly public government — transparency leading to collaboration. I am trying to come up with a simple taxonomy of transparency, a list of what should be open by default. Help me with my definition and list of buckets:

* * *

The first step to public government is transparency. My definition: opening up the information and the actions of government at every level by default in a way that enables any citizen to take, analyze, and use that data, extracting or adding value to it and overseeing the actions of those who act in our name, with our money. That data should include:
• Our laws and regulations—as they are being considered and after they are enacted, showing who did what to each along the way.
• Government budgets and spending, including information on who is paid.
• Government’s actions. I want to see crimes, complaints, actions, conferences and other events, even useful correspondence.
• Government information of every sort. The New Republic says making weather data public “produces more than $800 million in economic value.” Global positioning data enabled the creation of now-indispensible smart phones and navigation systems. Agricultural data saves and makes money on farms. If the government knows it, we should know it.
In What Would Google Do? I joked—well, half-joked—that the Freedom of Information Act should be repealed and turned inside out so that we no longer have to ask government to open up our information; government must ask our permission to keep it from us—especially now that technology gives us the tools to make use of it. There are a legitimate reasons not to release data: because it reveals personally identifiable information about citizens (e.g., your tax bill—though, again, that’s published in Scandinavia) or it compromises security or criminal investigations. Other than that, our information is ours and so we need access to it. In our developing information economy, this data has real and growing value. So cough it up.

* * *

Of course, the discussion is all the more timely thanks to Wikileaks’ latest deluge of once-secret data. The further question is how anything can be held in secret and what the appropriate line is for secrets. In the midst of the last Wikileaks disclosure, I suggested that the only cure for leaks is transparency: when the public trusts that a secret is secret for good reason, then revealing it is more clearly a violation of norms and the common good. But when most government actions and information are held from us, then exposing them is more just but will bring the collateral damage of also exposing things that should properly be held in secret. Whom do we trust with that judgment? The government? Wikileaks? The Guardian and Times? That’s what is being wrestled down right now.

In any case, I’d appreciate any help with the organization of thinking around what’s properly public and not. Thanks.

Transparent inventory & the rebirth/death of retail

The Times reports this morning on smart retailer Nordstrom making its inventory in warehouses and in stores transparent so a buyer who’s dying for a purse can find it nearby (for immediate gratification), or from the warehouse (for convenience), or at the last store that has it (which will ship to her).

The earlier rendition of this was BestBuy or B&N making it possible for customers to find whether an individual store had an individual item via their web sites; that’s not quite as easy as saying, “wherever it is, just get it to me,” but it was an important step in this direction.

The next rendition of this will be, I think, enabling the customer to search across multiple retailers and order. That will give us what Dave Winer tweeted this morning that he wants: “I wish there was an Amazon store in midtown Manhattan, where I could buy anything that is available for same-day delivery. I’d go there now.”

Well, if Amazon did that, it would be gigantic, expensive, capital-intensive, inventory-filled Wal-Marts peppered all across the country; it would be expensive to build and it wold lose the efficiency and profitabliy that Amazon enables.

But the virtual version of what Dave wants is possible with transparent — and open — inventory, enabling customers to ask, “Who has this item nearest me at the best price? Then I’ll go get it or get it delivered to me today.”

Ah, price. There’s the rub, of course. Such a network would make pricing transparent. It pretty much already is. We can search across individual retailers or use the likes of Froogle and get the lowest prices. There lies the downfall of retail’s margins, taking out the ability to arbitrage opacity in pricing. Now add transparent inventory and the ability to get the item at the lowest price now and the value of retail brands sinks as does its profitability — in an already tough-margin business.

I could imagine a new service — from Amazon or an entrepreneur — that says: Tell me what you want, Dave, and I will tell you where you can find it or I’ll get it to you. That’s the new value-add for those who want to touch an item or get it immediately. The other value-add is support and service (see: Best Buy’s Geek Squad). Putting boxes and shelves and waiting for people to buy them while carrying the cost? That no longer adds value; it only increases risk.

Retail is going to get ever-more efficient. Independent bookstores were killed by the more-efficient box stores. Box stores were wounded by Amazon and the internet. Amazon could be injured by a local value-added retail search-and-delivery service. All this is fine for the customer: more choice more quickly at lower prices.

But I think retail could be headed the way of newspapers: into a pool of endless pain. All over America, I see empty retail shells: the former Circuit City, the closed book box, the folded mom-and-pop. I think there’s much more of that to come.

This is what transparency — in price and inventory — can do to a market.

What if there are no secrets?

Is no secret safe?

That’s the moral to the Wikileaks war log story: you never know what might be leaked. Of course, that itself is nothing new: Whenever we reveal information to even one person, we risk it being spread. The ethic of confidentiality (and privacy) rests with the recipient of that information.

So what’s new now? There are more means to get information since it is pooled and digital. There are more means to share information; Daniel Ellsberg had to go through media to spread his Pentagon Papers while Wikileak chose to go through media so they could add value (perspective and attention) but didn’t have to. And there are new means to stay anonymous in the process.

I’m writing a book arguing that we are becoming more public and that’s good — and that institutions (government, companies) have no choice but to live up to our new standards of transparency and openness. But I am also examining when transparency goes too far.

Is the Wikileaks story an example of crossing a line? First, we have to ask where the line should be. I think it has to move so that our default, especially in government, is transparency. Rather than asking what should be made public we should ask why something should be kept private. Imagine if all government information and actions were public except matters of security and personal and private identification. There will be pressure to head there.

I make the mistake of thinking that we’ll navigate toward openness via rational and critical discussion. But we’ll more likely move the line because of purposeful subversion of the line like Wikileaks’. The line will be moved by force.

Now that they’ve made the war log public, it makes us examine the impact.

We need to ask whether the knowledge that anything written down could be made public will cause less to be written — and we lose information in the long run. That is my concern about efforts to make *all* government communication, including person-to-person email, permanent and public. I imagine that people will stop saying important things in email and instead pick up the phone and we lose the record.

We need to ask whether an ethic of transparency can be expected when leakers can be anonymous and their leaks swift.

We need to ask whether the government would have been better off making more public so that the leaker’s selective publication does not solely set the agenda and the government is stuck reacting.

In the war logs, we are learning things we should know. It’s the leakers — Wikileaks and its three media outlets — who are deciding what not to make public (with some consultation, post-leak, from government) and what should be open. So government loses the ability to decide secrets. Now leakers do. Which side do we trust to decide?

The sane response to leaks, I think, is to open up as much as possible. Then there’s nothing to leak except the things that shouldn’t be leaked. If we had the faith that we knew more, there’d be fewer leaks, fewer reasons to.

I don’t think this is an inexorable process of opening everything, of making no secret safe. As much as I advocate transparency, I don’t advocate that. But when you don’t know how many secrets there are, when there are too many secrets, then everything can be a leak — in Afghanistan or in the Gulf of Mexico. Unless government and business take on a credible and complete ethic of transparency, they will hand over the job of transparency to leakers and no secret is safe.

ANOTHER THOUGHT: This story is a step to the end of access journalism. (NO, it won’t end. Whenever people like me declare the death of something, disbelieve and discount it; we’re just saying we’re heading away from something).

But Wikileaks didn’t need, doesn’t want, won’t ever get official, journalistic, beat access. The derailing of a general in Rolling Stone didn’t come from a beat reporter who cared about access anymore. ProPublica’s work isn’t built on access.

When I talk about how little is spent on investigative reporting in America — as a proportion of total editorial spending across all media, it’s minuscule … microscopic — editors remind me that my calculation doesn’t include beats and beats are the heart of reporting. True and not true. It’s true to the extent that we want ongoing coverage and want it performed by people who build up experience if not expertise in the subject. It’s not true to the extent that reporters who depend on access from the subjects won’t ruin the relationship by breaking the subject’s secrets or the access (and the reporter’s supposed value) ends. (This is why reporters aren’t supposed to blog their opinions about their beats, according to fresh orthodoxy: They would lose access.)

In access journalism, leaks come from the subject. In unaffiliated journalism, leaks come in spite of the subject. As more reporting is done through mechanisms like Wikileaks and ProPublica and bloggers and advocates, I think we’ll see more breaking of secrets, which reinforces my point above: the best way to fight leaks is transparency (not black-out paint).

The Pentagon learned that lesson just a bit when it realized that giving more access would mean more control. Thus the embedding program in Iraq and Afghanistan. But news organizations can’t afford to have reporters embedded in the war zone. Coverage was too dependent on relationships. That honeymoon is ended.

The coverage of this war revealed much of what we know from the war logs. Alex Thomson says, though, that the logs validated what we know. They added facts we couldn’t get with access.

As news organizations shrink, we’ll be able to afford less access journalism — fewer beat reporters building relationships with their subjects — and more reporting — and subversion — from people who have a viewpoint and an agenda. The tone and means of journalism changes. It becomes more uncomfortable. But then, isn’t journalism supposed to be uncomfortable?

: MORE: Many notes from Jay Rosen here: “I don’t have the answer; I don’t even know if I have framed the right problem.”

Jay talks about stateless journalism. Dave Winer says the blogosphere is that. I don’t think the issue is that journalism is stateless but instead that journalism is becoming independent of organizations (pace Clay Shirky). Journalism lacks affiliation. Anybody can feed WIkileaks; Wikileaks can feed anybody. The organizational — nevermind state — point of control disappears. Journalism is everywhere and its up to the public to decide what news is.

Though from another perspective, stateless does matter as we’re seeing more of it across many sectors of society. Our enemy in this war is stateless. Businesses are stateless. Journalism now becomes stateless. I believe the tools of publicness — that is, the internet’s — enable us to organize new societies around states.

: MORE: Andrew Potter breaks down the discussion into four questions.

The public life

The Guardian asked me to write a column about the transparent life and my writing about my prostate cancer. Here it is:

* * *

In the company of nudists, no one is naked and there is nowhere to hide. In this space and on my blog, I have been arguing that with the internet, we are entering an age of publicness when we need to live, do business and govern in the open. So I was left with little choice when I learned I had prostate cancer. I had to blog it.

So far, no regrets. Oh, one troll tweeted that in my blog post, I had merely used my cancer to plug my book (which, by the way, is entitled What Would Google Do?). But my Twitter friends beat him up on my behalf. I got emails pushing nutty cures on me – yes, there is cancer spam – but Gmail’s filters killed them for me. And I have had to be mindful not to bring my family into my glass house; my transparency shouldn’t necessarily be theirs.

But it has all been good. On my blog, on others’, in Twitter, and in email, I received an instant and lasting shower of good wishes and some good advice about my choice of surgery. My brothers in malignancy have shared their experiences with generous candour. I even inspired a few of them to blog their own stories. They joined me in urging men to have the PSA blood test that revealed my cancer.

After my blog post sharing the diagnosis was republished last week in the Guardian, I heard from Emma Halls, chairman of the UK Prostate Cancer Research Foundation, who said the disease affects almost as many men as breast cancer does women, but it gets less funding and little attention.

That stands to reason. We men don’t like talking about penises – certainly not when they malfunction. Discussing one’s incontinence and impotence post-surgery – both temporary, we hope – well, it doesn’t get much more transparent than that. It’s one matter for me to disclose my business relationships, politics, religion, and stock ownership on my blog’s “about” page; it’s another to do this.

So I think I’ve become about as transparent as a man can. I am living the public life. There are dangers here. I risk becoming merely a medical and emotional exhibitionist. And I know I have violated my own privacy to an extreme.

But I think we need to shift the discussion in this era of openness from the dangers to privacy to the benefits of publicness. It’s not privacy that concerns me, but control. I must have the right and means to keep my disease secret if I choose.

By revealing my cancer, I realise benefits, and so can society: if one man’s story motivates just one more who has the disease to get tested and discover it, then it is worth the price of embarrassment. If many people who have a condition can now share information about their lifestyles and experience, then perhaps the sum of their data can add up to new medical knowledge. I predict a day when to keep such information private will be seen by society as being selfish.

Collectively, we will use the internet’s ability to gather, share and analyse what we know to build greater value than we could on our own. That is the principle of transparency that I want companies and governments to heed: that openness in their information and actions must become their default, that holding secrets only breeds mistrust and robs them and us of the value that comes from sharing.

I believe this openness at the source will become a critical element in a new, linked ecosystem of news, as institutions and individuals will be expected to provide maximal information on the web. Such open intelligence also allows an unlimited number of watchdogs on those in power, helping to bring about a new, collaborative – and ultimately, I hope, more effective and efficient – system of journalism.

So for me, transparency is a necessary ethic of the age. That is why I used my medium, my blog, to share my prostate cancer. If I believe in the value of publicness, how could I not?

The small c and me

I have cancer, prostate cancer.

When the doctor told me, he said that if you’re going to get it, this is the one to get. It made feel as if I’d just gotten an upgrade on Cancer Air. It was caught very early, found in only 5 percent of one of 12 samples gathered by shooting a harpoon gun into me (where, you don’t want to know). So I am lucky.

I’m reminded of a brainstorming session I went to with Tony Hendra, the comedy writer, toward the end of the ’80s, when he was leading the collaborative writing of a book called The ’90s: A Look Back. I was invited to a session where we speculated about the near future of medicine and Tony riffed about what it would be like once they found a pill to care cancer. “Got a spot of cancer today?” he said, copyrighting. “No problem. Take Tumorout. You’ll feel as good as new. Go ahead. Light up that cigarette. Won’t hurt a bit.” I was disappointed that his cancer gag didn’t make it into the book. I’m also disappointed that they didn’t invent Tumorout.

Why am I even telling you about this? As I wrote in What Would Google Do?, I gained tremendous benefit sharing another ailment – heart arrhythmia – here on my blog. And so I have no doubt that by sharing this, I will get useful advice and warm support (and maybe a few weeks’ respite from trolls). I argue for the benefits of the public life. So I’d better live it.

I also hope to be one more guy to convince you men to get get your PSA checked: a small mitzvah in return for my luck. And when we talk about the cost of screening in the health-care debate, I’ll stand up to say that when you’re the 1-in-100, screening is worth it.

I’ve always been a cancerphobe; can’t imagine much worse than that creeping invasion. Yet I’ve surprised myself, staying calm in the face of realizing my fears, probably because I know it could be worse and, well, it is what it is. I’ve been using this amazing internet to do research and, with my wife’s help and counsel, make the complicated decision on a course of treatment.

Before doing my research, I’d assumed that the treatment Rudy Giuliani made famous – radioactive seeds – would be the way to go: simple, and if it doesn’t work, I thought, then I could resort to surgery. But it turns out that once you get zapped, it becomes very tricky to perform surgery. At my age – young, damnit – the wiser course is surgery, cutting out the prostate and, one hopes, all the disease with it.

I’m opting for robotic surgery – geek that I am, how could I not? My only fear is that they’ll wheel me into the O.R. and I’ll see that the machine is powered by Dell.

I’ve also chosen Sloan Kettering and Dr. Raul Parra to do the surgery. There’s one of the privileges of living in New York, among the best.

I’ll keep you informed as I find notes of interest while progressing toward surgery in mid-September and through recovery. Fear not, I’m not going to turn this into a disease journal: I don’t expect you to be consumed with my problems when others have theirs, far worse. Or perhaps you should fear, for instead, I will keep on writing about media wonkishness: about the rise of the next media and the fall of the last. Except now, I’ll be in a worse mood.

WWGD? brought to life

There’s little I love more these days than seeing people bring the precepts of What Would Google Do? into their realms. I just hope I’m right and don’t lead them astray.

Here‘s a post by Arild Nybø Førde, a Norwegian entrepreneur who wonders whether he’s better off being transparent about his business idea, weighing the input he’d get against the risk of someone taking is idea:

Why keep secrets?”, Jarvis asks, and he rephrases: “Why keep more secrets than you have to?” The most common answer is of course that we don’t want our potential competitors to steal the ideas. That would also be my answer.

But is the risk of being lifted for ideas greater than the risk of failing by not being transparent? If asked myself this question, and after some consideration my answer is “No”. . . .

OK, so having great ideas is not my biggest challenge right now. It’s the ability to evaluate them, and then realize them, which will be my greatest struggle in the months and years to come.

So, how can being transparent help the ability to realize the ideas? . . .

Right now I’m alone with my ideas. I don’t have any employees yet. . . .

Therefore, I guess, the best way to get my ideas analyzed and criticized, is to let them out in the open. And what is more open than the World Wide Web?

In addition to getting feedback to my ideas, through blogs and social media, laying them out publically will also kick my butt. As long as I write in my blog, or state in an interview, that I’m going to do this and that, I make a commitment to my readers. . . . And if I don’t fulfill my commitments, I can’t be trusted, and my business will fail.

That’s why I’ll risk it. From now on I’ll be transparent and reveal more and more of my strategy in this blog.

And then there’s this exec, writing under the name Oz, talking about how cable should be run today:

If I were running the cable company I would do pricing and product offering differently. Transparency and simplification will be a core objective. Reducing the barrier to taking on new clients/customer will be at the fore of every decision and Internet copy that is produced. Borrowing from Jeff Jarvis in What Would Google Do? Simplify or die. This should be a pillar of running any business in the age of the web. Firms should no longer seek to profit from cordoning off a section of the market hoping that the black box of disinformation which they have built will not be found out and exposed.

I am sure some analysts will be quick to suggest that this disinformation that I speak of is a cash cow. Meaning that this firms make money from their ability to coax new clients into paying more than they usually should. To me, this is an old world way of thinking, information wants to and will be free, any business model built on erecting barriers to information will eventually fail. This idea may have served the firms well in the past but with many more firms, channels and platforms competing for my attention, it is a strategy headed for disaster.
Competition will blindside these firms. Any firm with a good enough service that chooses to be transparent and simple will win. I would be the first to line up at their door.

PS: This applies to all firms in all industries.

Yesterday, I had lunch with a reporter-friend who can’t advise companies because he’s a reporter; he can only soak in, and not bounce back. But I said that I’ve learned so much talking with companies – in person or on this blog – to learn their problems and their opportunities as they try to reinvent themselves – as the good ones are – for our new reality. It gives me the chance to test my ideas and observations against their reality and there’s nothing more valuable than that. As Arild said above, transparency and interaction are what enable analysis, criticism, challenges, and improvements.

The start of transparent government

The announcement of Data.gov marks an important shift in government, opening up our data to us and enabling collaboration and creation with government.

Jake Brewer of the Sunlight Foundation also announces a contest to create apps atop the API.

I believe that in the future ecosystem of news, transparent government data will play a key part. It will enable us to have millions of watchdogs on government’s action.

I also hope that this openness starts to shift the conversation around government from get-the-bastards to collaboration and creation.

Brewer says:

New federal CIO Vivek Kundra and the Obama Administration have officially launched Data.gov, which is the first-ever catalog of federal data being made freely (and easily) available to citizens.

Now, it’s unlikely the description of Data.gov will send chills down the spine of anyone who doesn’t speak Ruby or Python or MYSQL, and if you visit the site, it’s unlikely you’ll be struck or know to be impressed by what’s there. But if you step back and take a minute to understand what you’re looking at, you’ll realize we’ve just taken an unprecedented first step into the Era of Big Open Government.

When information and process become free and participatory, markets get created (think about weather data), more people engage more deeply with their government (see: Obama’s online townhall), and ultimately, people care more about what their government does and how it serves them. …it’s nearly impossible for people to know more about what’s going on and care less.

Transparency is at the heart of destroying apathy.

The key with this new data, though, is that we do something with it. While opening up data is a beautiful thing in its own right, what will make this release truly great is when citizens actually take the information and create new, brilliant applications.

That’s why Sunlight Labs in partnership with Google, O’Reilly Media, and Craig Newmark of Craig’s List has simultaneously launched a contest with $25,000 in awards to incentivize the creation of said brilliance.

Apps for America 2: The Data.gov Challenge