We must not enable encryption backdoors in consumer products

As the Apple encryption controversy rages on, here's my take on it.

There should never be backdoors put in consumer products to enable the manufacturer or the government to bypass encryption protections. (It is however, reasonable for the government or an enterprise to enable a feature like this for their own devices that they manage and provide for their own internal uses.)

The obvious question is then Why? Doesn't this enable bad people to do bad things?

Look at it this way. You understand how your home's door lock works, right? You have a key that opens it. You also probably understand how a locksmith with the proper knowledge can bypass your particular lock.

Now, imagine that your home's lock has a "master key" feature created by the manufacturer. If you possess this master key, you can unlock the door to anybody's home that uses that brand of lock. These master keys are carefully controlled and distributed to licensed locksmiths only. The law says that only licensed locksmiths may use master keys and only under court order.

Seems pretty safe and reasonable, right? Only the good guys have the master keys and they use them for the right purposes.

Now, imagine that just *one person* out of all 7 billion on the planet is able to fool the manufacturer of the lock into believing that they are one of these trusted locksmiths. They end up with a master key in their hands. Doesn't seem too bad yet, right? It's only one person and laws prohibit them from using it.

Next, imagine this person scans the key and produces a 3D printer model of it. Then they put that model out on the Internet for anyone to download. Now, any criminal, anywhere with access to the Internet and a 3D printer can obtain their own master key for that brand of lock with a very small amount of work.

Do you think this is a far-fetched and unlikely scenario? Think again.

The exact same reasoning applies with the situation Apple is discussing. The only thing preventing all backdoor enabled devices from being decryptable is a little bit of extremely sensitive information stored in the hands of a few good (and imperfect) people. Once that information gets out – and with the ever-increasing number of serious cyber security breaches (see Edward Snowden and Office of Personnel Management), it is very likely that it eventually will – every single device in use becomes instantly decryptable.

That is just too dangerous of a situation to allow. It will instantly endanger people everywhere in sometimes life-threatening ways. And I've got good company in taking that position. The present and two former NSA Directors agree:

The US is “better served by stronger encryption, rather than baking in weaker encryption.” 

Yes, we will be protecting criminals and their behavior at times. But so do the 4th and 5th Amendments to the Constitution.

The danger of the alternative is too high. As H.L. Menken put it:

The trouble with fighting for human freedom is that one spends most of one's time defending scoundrels. For it is against scoundrels that oppressive laws are first aimed, and oppression must be stopped at the beginning if it is to be stopped at all.


Demographics and the 2010 U.S. Census

One of the pleasures of developing custom software through my company Data Bakery is the regular opportunity to learn about new and interesting topics. In the case of our client StreetCred, that topic turned out to be demographics and how they were handled in the 2010 U.S. Census. Demographics are an important topic for StreetCred because their software helps law enforcement organizations (LEOs) understand how their officers are interacting with the communities they serve.

The way race and ethnicity are handled when dealing with demographics tends to be fuzzy and happenstance. However, in the 2010 Census, the U.S. Census Bureau decided to be much more explicit: they treated Hispanic as an ethnicity separate from race (White, Black, American Indian and other races). This decision has resulted in enough confusion that the U.S. Census Bureau issued a 23 page document titled Overview of Race and Hispanic Origin: 2010 in March of 2011 that attempts to explain the situation. Briefly summarized: Hispanic is considered an ethnicity and not a race. In other words, Hispanics are in the majority considered racially White.

As a result, the question of whether or not a respondent was of "Hispanic origin" was asked independently and in addition to the respondents race. This may seem like just an interesting definition of little consequence, but the way this was handled has a direct impact on consumers of Census data who are more interested in answering demographic questions where ethnicity and race need to be considered together.

A Maze of APIs and Variables

When you first begin exploring the U.S. Census data, it comes off as an arcane and maze like combination of datasets and APIs. That tends to lead people to the US CitySDK which was created by the U.S. Census Bureau to make it easier to access U.S. Census data. Unfortunately, the CitySDK is designed to run in a web browser which limits how it can be used. However, it does demonstrate how and which U.S. Census APIs to use for various types of queries. In particular, demographic queries in CitySDK use the American Community Survey 5 (ACS5) dataset.

While the ACS and 2010 U.S. Census are two different surveys/programs, the data surveyed and reported in the ACS is handled the same way as was decided for the 2010 U.S. Census – Hispanic is an ethnicity and not a race.

As you dig deeper into the ACS5 variable set and experiment with it, you begin to get a sense of how the data is structured. Consider the following variables:

Variable Name Label
B02001_002E White Alone
B02001_003E Black or African American alone
B02001_004E American Indian and Alaska Native alone
B02001_005E Asian alone
B02001_006E Native Hawaiian and Other Pacific Islander alone
B02001_007E Some other race alone
B02001_008E Two or more races
B01003_001E Total

Sounds pretty good so far, right? If you pull the data for Fort Worth, here's what you get:

Label Count
White Alone 508,894
Black or African American alone 147,471
American Indian and Alaska Native alone 4,621
Asian alone 28,984
Native Hawaiian and Other Pacific Islander Alone 1,104
Som other race alone 65,096
Two or more races 22,403
Total 778,573

Everything seems good and the math adds up. However, what if you want to know what proportion of your population is Hispanic? You can see some of the confusion on this if you look at the Wikipedia entry for Demographics of Fort Worth.

If you look further into the ACS5 variables, you find:

Variable Name Label
B03001_003E Hispanic or Latino

If you add this to the ACS5 query, you get 266,472 for Fort Worth. Nice to know, but what do you do with this number? You can't add it to the counts above because that would be over-counting people. The only way to combine this ethnicity count with the racial counts would be to reduce the counts above to just the people who answered a particular race but also said "no" to Hispanic origin. But it is impossible to determine how to do this from just the numbers above.

The Lesser Traveled Path

Luckily enough, if you dig further into the ACS5 variable set, you find these:

Variable Name Label
B03002_003E Not Hispanic or Latino – White Alone
B03002_004E Not Hispanic or Latino – Black or African American alone
B03002_005E Not Hispanic or Latino – American Indian and Alaska Native alone
B03002_006E Not Hispanic or Latino – Asian alone
B03002_007E Not Hispanic or Latino – Native Hawaiian and Other Pacific Islander alone
B03002_008E Not Hispanic or Latino – Some other race alone
B03002_009E Not Hispanic or Latino – Two or more races

With these variables, you can now query the following data:

Label Count
Hispanic or Latino 266,472
Not Hispanic – White Alone 318,732
Not Hispanic – Black or African American alone 145,330
Not Hispanic – American Indian and Alaska Native alone 2,262
Not Hispanic – Asian alone 28,534
Not Hispanic – Native Hawaiian and Other Pacific Islander Alone 966
Not Hispanic – Some other race alone 1,228
Not Hispanic – Two or more races 15,049
Total 778,573

And there you have it. Demographic data combining ethnicity and race using post-2010 U.S. Census data.

Part 3 - The DAE Paradigm Shift

In Part 1 we discussed the problem of reversible application encryption, and, in Part 2, we presented the details of Distributed Application Encryption (DAE), a proposed approach for application-level encryption designed for our current security environment. In Part 3, we cover the pros, cons, and other implications of DAE.


  • Encryption Diversity – each user has their own encryption key for their own sensitive data. In order to obtain all data in plaintext, you need to gain access to each and every user’s unencrypted User Master Key. Put another way, cracking the key for one user doesn’t affect the security of another user’s sensitive data.
  • Narrow Vulnerability Windows – the User Master Key is only exposed during user requests. Likewise, the Unlocked Session Cookie only exists while the user is logged in.
  • Geographic Diversity – Unlocked Session Cookies are stored with users’ web sessions, not on the server. An attempt to collect them on demand would require attacking many different users devices simultaneously.
  • Restricted Attack Scenarios – Most attack scenarios involve modifying an application to collect plaintext information. Under DAE, an attack of this type would expose only those user accounts accessed from the point the attack begins until it is finally discovered. Theft of an entire database in plaintext form at any particular point in time would be impossible.
  • Transparency and Accountability – Applications that need to access sensitive data must request that users return to the application, establish a session, and make a request to provide the needed Unlocked Session Cookie. This provides transparency for users as to how their sensitive data is being handled, as well as creating an implied-approval process of that handling for the application provider.


  • DAE is dependent on a plaintext password sign-in mechanism in order to unlock data.
  • Applications implementing DAE are susceptible to code-modification attacks in which surreptitious collection of User Master Keys is added in the request path.

Regardless of the technical pros and cons, adopting DAE imposes a new paradigm for managing sensitive data – a paradigm that requires fundamentally rethinking how applications handle sensitive data.

Applications Should Only “Borrow" Sensitive Data

The new paradigm can be summed up as follows: a DAE application neither owns nor has free access to sensitive data it collects from its users; it must always ask for and be granted that access. To benefit from DAE, application designers and implementers must embrace this principle and implement it elegantly and thoroughly.

A real-world analog to DAE is a safe-deposit box at your local bank. You store valuables in it, and neither you nor the bank can access its contents without mutual coordination (barring court orders, thievery, etc.). This mutual coordination is guaranteed via an access system requiring two keys – yours and the bank's – to unlock the safe-deposit box.

This parallel also highlights the major drawback: valuables are less accessible for both the user and the bank.

For some applications and some types of sensitive data, converting to this information-management system would be simple. For instance, a doctor’s office rarely needs a patient’s Social Security number. Storing it and requesting access to it via DAE seems reasonable for both doctor and patients.

But imagine a financial application that handles sensitive data like account numbers. Many types of transactions (interest accrual, check clearing, bill pay, direct deposit) would prove cumbersome if each required an access request to the customer. There will likely need to be per-user, per-application, and even per-industry analyses and standards establishment as to what is treated as “sensitive data” in a DAE architecture.

DAE variants are also a possibility. Users could grant temporary server-side storage of their Unlocked Session Cookies in order to handle batch and other types of autonomous processing. Application designers might unlock sensitive data and temporarily store it in plaintext for batch processing runs. There are bound to be many other variations possible. 

It’s a Process, Not an Event

This series of posts was not intended to define and promote DAE itself. DAE is merely a useful vehicle for defining and promoting the paradigm shift it represents. The real measure of DAE's worth will be apparent in how this paradigm shift is regarded and incorporated into application design in general. While this discussion has assumed a web application environment, the techniques described could apply in many scenarios.

Reasonable technical minds can differ on what exact form the paradigm shift should take. But one fact none can deny is that, as each new data breach makes headlines, it becomes ever more painfully obvious that our current approaches aren’t working.

Part 2 – Distributed Application Encryption (DAE)

In Part 1 I talked about the ready reversibility of application encryption. At the root of that problem is how encryption keys are managed. In most applications, encryption keys must be readily available so the application can decrypt and encrypt any of its data at any moment. When this availability is combined with a successful spear phishing attack on an administrator, it is only a matter time before any or all application data is decrypted and exfiltrated.

In direct contrast, DAE defends much more effectively against this threat scenario. Black hats can have complete run of your infrastructure and yet still not be able to decrypt sensitive data. DAE achieves this with a trade-off: ready accessibility vs. encryption durability.

While typical application level encryption stores a handful of encryption keys in config files or HSM modules and uses them in different functional areas of the application and database, DAE manages one encryption key per user. This means that, rather than organizing encryption by functional or database area (for instance, by protecting all Social Security Numbers in the app or targeting columns of a particular table), encryption is organized and employed on a per-user basis.

When a user is added to the system, a random encryption key (the User Master Key) is generated for that particular user by the server. This User Master Key is used to reversibly encrypt and decrypt any sensitive data related to that user. When the User Master Key is at rest, it is encrypted with the User Session Key. The User Session Key is derived from the user's plaintext password when it is presented during account creation or later during sign-in.

Operationally, the User Master Key is “unlocked” whenever a user signs in. During sign-in, the user presents their plain text password to the server, the server generates the User Session Key and decrypts the User Master Key. At this point the User Master Key is now encrypted with an Application Master Key (which itself is stored in a config file) and stored as an Unlocked Session Cookie in the user’s web session. The User Master Key is now “unlocked” while the plain text password and the User Session Key are discarded.

Each time a user makes a request to the server, the user’s session provides the Unlocked Session Cookie with the request. If access to that user’s encrypted data is needed, the server can decrypt the Unlocked Session Cookie with the Application Master Key to get the User Master Key. The User Master Key can then be used to encrypt or decrypt the relevant user data. When the request is completed, the User Master Key is discarded. When the user’s sign-in session ends, the Unlocked Session Cookie is removed from the session.

The following diagram goes into these scenarios in greater detail.

Distributed Application Encryption Use Case Scenarios

Distributed Application Encryption Use Case Scenarios

In Part 3 we’ll discuss the pros, cons and other implications of DAE.

Part 1 – The Security Elephant in the Room

If you keep up with technology, you know major cyber security breaches have become commonplace. From Target and Home Depot to the US Office of Personnel Management, these break-ins are growing larger in scope and happening more frequently. While everyone acknowledges data theft is troubling, even security professionals just shrug it off as business as usual.

However, I believe a subtle but significant shift has occurred in the security landscape. If you build any sort of web/Internet system, the chances that your product's entire database will be stolen sometime during its lifetime are now close to 100%. Once you acknowledge this reality, new, worrying questions arise about how to protect application data – and what kind of liability companies may face if they continue to build applications without taking this reality into account.

The Placebo Effect

If you talk with security experts, they will tell you that the answer to our computer security problems is application-level encryption. While application encryption approaches are appealing, they have one serious flaw that these experts tend to hand-wave away: ready reversibility. Granted, application-level encryption provides general improvements in diversity, opacity and at-rest protection. However, with the right administrative access, it is usually trivial to reverse these protections. This makes the whole approach vulnerable to one of the most challenging security threats today: spear phishing.

Let’s say you develop a web application employing multiple encryption keys to protect data with multiple algorithms at multiple different points in your database. Sounds robust, right? At least until I spear phish one of your dev ops guys and use his administrative access to exfiltrate the source code to the application, its database and the keys that protect it all.

Oh, but you are using a Hardware Security Module (HSM) and are protected against this scenario? Well, all I have to do is be patient and use your dev ops guy’s access to all of the involved systems to run the data through the HSMs and decrypt it before exfiltration. Or even worse, I discover and use maintenance functionality (e.g. an “archive” function) already built into the application to get it to export all of its data unencrypted.

Houston, We Have a Problem

Once someone has administrative access to your systems, most encryption techniques are easily reversible. Thus, since it is likely impossible to keep black hats from penetrating your infrastructure and gaining administrative access, security that relies primarily on application-level encryption is fundamentally flawed.

What’s the answer? In Part 2, I’ll propose a new approach to application encryption which I call Distributed Application Encryption (DAE). In Part 3, I’ll discuss the strengths, weaknesses and implications of DAE.

The Password Manifesto

I'm not an active member of the computer security community, but I have considerable knowledge of and experience with the topic. I've been thinking about and working on the password problem for many years now and have come to some conclusions that I want to share more widely.

(This document is still in draft form. Please suggest enhancements in the comments, on Twitter or contact me with your thoughts or if you are interested in becoming a signatory when complete.)

#1 – Attempting to eliminate passwords as an authentication method will never succeed due to market inertia. The industry should instead focus on evolving password authentication.

Password authentication is the most common authentication technique and is understood by a large number of users and by a large number of developers. There are very large amounts code that have been written that implement password authentication. Countless numbers of systems rely on it today.

#2 – It is time to acknowledge that knowledge factors are no longer viable and should be removed from online authentication systems.

Authentication is normally accomplished by various combinations of factors: something you know (knowledge factor), something you possess (possession factor) and something you are (inherence factor). Knowledge factors are problematic for users who struggle to manage them. This situation has been compounded as password complexity requirements have increased and hacker techniques have evolved.

#3 – All passwords should be completely random, consist of 256 bits of entropy derived from  cryptographic random number generators and be assigned by service providers instead of chosen by users. This transforms the password from a knowledge factor into a possession factor (i.e. a key).

Computing power and cracking techniques have made effective passwords almost impossible for humans to represent as knowledge. Therefore, it is time to eliminate knowledge factors as a class and focus on possession and inherence factors.

#4 – The W3C should define a "Keychain" API that allows website Javascript to store keys in a user's local keychain during account setup and retrieve keys during authentication.

Web browsers and third party password managers attempt to detect login forms in order to automate password authentication. This is difficult to implement and error-prone.

#5 – All keychain implementations must encrypt and decrypt key content locally. Vendors must not be able to access key content by design. Any key content that leaves the context of a device must be encrypted such that only the user can decrypt it via local authentication methods.

Vendors and service providers should assume that their infrastructure will be penetrated by hackers at least once in the lifetime of any product. Therefore, the only way to protect user content is to make it inaccessible to anyone but the user on their own device using local authentication and decryption methods. This is somewhat obvious today and many password managers are designed this way – but it should still be said.

#6 – All operating system and browser vendors should standardize on keychain formats and define standard APIs and formats for interchanging keys and other related content. Vendors should also synchronize keys between users' keychains on users' devices residing within each vendor's ecosystem.

Password managers are not the answer; password management is. Usernames, passwords and authentication are features of applications and operating systems – they are not a standalone product. They are best implemented and delivered as infrastructure. Vendors should recognize this and take responsibility for making it happen.


The Password is Dead, Long Live the Password!

I spent some time over the past couple of years working on technology to replace passwords along with a lot of other folks (LaunchKey, Clef, Nexiden, Google AuthenticatorSQRL, Mozilla Persona, oneID, etc. – the full list is sadly much longer than this). These ideas each have their technical merits and some get quite a bit of fanfare on their introduction to the market. However, I've personally reached the conclusion that none of these approaches are ever going to succeed in replacing the password with some nifty new authentication mechanism. It's not that there's no desire for a good solution nor that one can't be built. It's a much more fundamental problem: market inertia.

Rome Wasn't Built in a Day

The first thing to recognize is that the username and password problem isn't and will never be a product oriented problem – its nature is that it is an inherent feature of other products: applications, frameworks and operating systems. It's like alternative file browser software for your desktop OS. Sure, you can build it, but it's not going to be easy to accomplish and the operating system vendors can quickly react to whatever your value proposition is. Compounding the situation is the wide adoption and usage of passwords, the amount of books and training material about them and the large volume of training and experience so many developers and users have with them.

The moment you start looking at the problem this way you begin to realize the enormity of the task a new product in this space faces. How do you build a product that can create new standards, prompt people to write new books about the techniques and convert developers to using a new product when it is unlikely to ever successfully be adopted by enough applications, frameworks and operating systems to justify it?

It's a bit like throwing a pebble into a large pond and expecting anything more than tiny ripples to come from it before it quickly disappears below the surface. 

You might say "but wait – isn't this really just the chicken and egg problem?". I.e. if you could only convince a critical mass of users and providers to adopt a new system, the product would surely succeed, right? My contention is that no, this is a different and additional problem. You could have the niftiest product in the world, get a significant amount of users and sites to adopt it and would still face this market inertia problem.

So if the problem is nearly intractable from a product development perspective, what is the answer? Is this as good as it gets?

The Future is Now

I believe we need to stop trying to get rid of the password and instead work with the market inertia to get the password to evolve into a better direction. 

So, does that mean password managers are the answer? I contend that password managers are not the answer but password management is. In other words, password managers as a product market are a dead-end for the same reasons explained above. However, password management as a feature of applications, frameworks and operating systems is the future. Think LastPass versus iCloud Keychain. When your competitor controls the playing field you are competing on you've already lost – you just don't know it yet.

So, what will password management of the future look like? Here's how I think the password will evolve:

  • Passwords of the future will no longer be memorized. They will be very large, completely random and securely generated. A standard will be published on their format and exactly how to properly create them.
  • The user will no longer choose their own password – the service provider will assign it upon account creation.
  • A new web browser Javascript API will be introduced supporting key chain functions. This will allow a website to store an assigned password in your browser's keychain and allow that same website to ask for your password. Access and control will use origin domain protections and user prompts.
  • The web browser will connect to a platform key chain service that will synchronize and keep your passwords available on all of your devices.

You might have had two thoughts while reading that. 1) That sounds a lot like API keys that developers use to access web services, doesn't it? Yes, that's exactly right.

And 2) Doesn't the browser already support client authentication with client SSL certificates? There are some similarities, but the big difference is that the existing client SSL authentication comes with all of the baggage of a PKI infrastructure. The implementation of it is really dragging those problems down into the browser space – and I believe accounts for its limited adoption. The approach here is bubbling up a client authentication solution from passwords.

The User Experience

So, what would this look like to users? Here's two thought experiments for the most common cases:


A user visits a website and fills out a form to signup for service. There's no username or password field. They submit the form and once the account is created, Javascript sent back to the browser invokes the key chain API to add a new account (i.e. save a password for the domain). The user is prompted to give a description for the item. The browser saves it to the platform key chain and the platform key chain synchronizes it with the cloud and all of the user's other devices.


A user visits a website's login page. Javascript on the page detects and invokes the key chain API asking for a password for the domain. The browser presents a window to the user showing the description for each of the stored passwords for the domain. The user selects one which is then returned via the key chain API call to the website's login page Javascript code. The page proceeds to authenticate and sign in the user.

A couple of basic benefits become apparent:

  • A user will never see or be prompted for a password again. Evolving passwords will effectively get rid of them.
  • Every single password created and used this way will be cryptographically unbreakable. The only way to get your password is to steal it from you.
  • The main authentication problem will effectively move from being between service providers and you to being between each of your device's and you. This problem is much easier to solve locally since biometrics become viable. Touch ID anybody?
  • Passwords of this type (large and random) can service to both authorize and identify a user. There's no need to have a separate username to identify an account. The password (or hash of it) can identify an account.

One of my metrics for when an idea is feeling "right" is when it has a real-world analog that is tried and true. In this case, think of locks and keys. You might think login pages are the locks and passwords are the keys. But realistically, the analog for what we have today is really a combination lock. You have to remember, manage and protect the combination. Anybody who can obtain it can open the lock.

In this evolved password scenario, key chains and passwords are created, managed and protected like real keys. A real key isn't likely to be properly duplicated by just looking at it and you aren't likely to guess it. It requires possession of the key (or a duplicate of it) in order to gain access.

In other words, passwords change from something you know to something you possess.

It's Not a Startup Opportunity

If you've read this far, you probably have come to the same conclusion I have – this is not a startup opportunity. This is a problem of standardization and adoption by major application, framework and operating system vendors. Since this is an evolutionary approach instead of a whole new method, it's easy to imagine practical and cost-effective steps that could be taken to get there.

I believe whether or not anybody actively plans this outcome, this is where the problem will ultimately evolve to anyway following the current course. But the description presented here isn't a complete solution as authentication is a large and thorny problem. What about password resets if my key chain loses a key? How do I access websites when I'm not on one of my devices? Feel free to chime in with your thoughts in the comments and add to the discussion.

It's not a problem a startup will come along and magically solve. It's our problem to solve right now.

Durus: The Best Python Object-Oriented Database You've Never Heard Of

I've been developing software in the Python programming language for over 20 years now. It's my preferred language due to it's readability, speed of development and massive number of modules available for it. This blog post is about one of the hidden gems in the Python world: Durus.

If you've built any reasonable sized application before, you've likely worked with a database. The most common database technology available is Structured Query Language (SQL) databases which takes a row and column approach to storing, querying and retrieving data. This technology has been around since the 1970's and has dominated in terms of deployments.

In recent years, new database technologies have been becoming more common and popular. These include key-value, graph, object and document storage systems. Together, this whole trend has been labeled "NoSQL".

Durus actually predated this trend having been developed in 2004 and presented at PyCon 2005. It has it's origins in ZODB which was developed in the late 1990s – Durus took the general architecture of ZODB and simplified it. While it was introduced to the world in the context of web applications, Durus is really more widely applicable than that. It's ACID and can be run standalone or in a client/server architecture for scalability.

If you aren't familiar with object-oriented databases, they are quite a departure from the SQL model. Databases don't consist of tables and rows – they are collections and objects. For instance, in Python, the most common mutable object types are lists, dictionaries and sets. These have direct counterparts in Durus that act and behave like their corresponding type – but are Durus aware.

NoSQL – The Python Way

There's no better way to understand Durus than to see it in action:

$ durus -c --file=test.db
    connection -> the Connection
    root       -> the root instance
>>> root
<PersistentDict 0>
>>> root.items()
>>> root[1] = "a"
>>> root.items()
[(1, 'a')]
>>> connection.commit()
>>> ^D
$ durus -c --file=test.db
    connection -> the Connection
    root       -> the root instance
>>> root.items()
[(1, 'a')]
>>> from durus.persistent_set import PersistentSet
>>> s = PersistentSet()
>>> s.add(1)
>>> root["set"] = s
>>> connection.commit()
>>> root["set"].add(3)
>>> root["set"]
<PersistentSet 32>
>>> list(root["set"])
[1, 3]
>>> connection.abort()
>>> list(root["set"])

Fundamentally, an object-oriented database is based on an object graph. It has a root object that refers to everything else that will be added to the database. The database connection is used to control transaction semantics. Containers, fundamental types and custom classes can all be added to the database as long as the object graph they are a part of is connected to the root object.

There is one "gun meet foot" pitfall to watch for: never put non-Durus aware mutable objects or containers in the database. I.e. you should never add standard Python lists, dictionaries, sets or subclasses of object to the database. The Durus versions are instrumented to properly capture changes in a commit if their contents are modified. Otherwise, you will lose data that you think is being committed to the database.

Why Durus?

Philosophically speaking, object-oriented databases are quite elegant and simple to work with. They use the language runtime environment itself to implement full database functionality without having to learn new semantics (i.e. SQL).

Need a new table? Declare a new Durus persistent class and add a Durus container to the root. Need to do a join? Write a nested for loop across two containers. Need a large-scale indexed container? Use a Durus BTree.

And you get all of this simplicity and elegance in a high performance package. Durus uses an append-only file format on disk (along with a packing utility) and a memory cache which enables large deployments: 10's of GB databases, millions of objects and quick access times.

So, if you got to the end of this post and still haven't installed and played with Durus, what are you waiting for?

Entrepreneurship – Dallas, TX Edition

If you Google "entrepreneurship", you get back this definition:

Entrepreneurship is the willingness to take risks and develop, organize and manage a business venture in a competitive global marketplace that is constantly evolving. Entrepreneurs are pioneers, innovators, leaders and inventors.

While there is a long history of entrepreneurship in certain parts of the world, the trend is picking up worldwide:

We are part of the global entrepreneurial class, an identity that transgresses borders, nationalities, and religion.  Entrepreneurs are a demographic, not a geographic, and their conspicuous creation is driving positive change in our world. Silicon Valley remains a bastion, and a gravitational force.  But the walled gardens are withering, and the access class is becoming an asset class.  Investors and entrepreneurs need a passport to the present.

The Dallas, TX area is no stranger to this phenomenon with organizations and activities like The DEC, Tech Wildcatters, The Garage, VentureSpur Texas, Common Desk, The Grove, BigDOCC, LaunchDFW, Dallas New Tech, DFW Startup Happy Hour, and plenty more. DFW has been no stranger to startup companies in the last 20 years either.

And of course there are local personalities helping driving this trend such as: Bradley Joyce, Mike Sitarzewski, Jennifer Conley, Michael GilbertTrey Bowles, and Gabriella Draney, to name but a few. And my small contribution is mentoring with a couple of the accelerator/incubator programs.

So far, it's a pretty straightforward set of facts repeated in various cities around the world, right? Not so fast, pardner – here in Texas we do everything bigger and better – and entrepreneurship is no different.

Meet Bill McNeely


Bill lives in Dallas and is a veteran of the Afghanistan conflict who, like many other veterans, has struggled since returning to civilian life. Between a moribund economy and the lingering effects of combat, Bill has struggled to support himself and his family. However, he's not just a veteran – he is also an entrepreneur and an active participant in the Dallas entrepreneurial community. This t-shirt he designed sums up his unique perspective:

Bill has been working on startup ideas surrounding the skills he excelled in with the military: logistics. The result is DeliverToMe, a B2B local delivery service. However, how he got to that point is just as important a story about Bill as it is about the Dallas entrepreneurial community.

Bill has received support from The Garage in acquiring a vehicle for his service and building his business model, acquired his first client Brevida, also a startup, through The DEC and received training and mentoring through the Google-sponsored Startup Weekend NEXT program led by Kevin Strawbridge where I helped Bill with his pitch and refined his business model. He also received in-kind support from FISH Technologies, a local award-winning experiential marketing technology company.

The details of how DeliverToMe has developed are much less important than how so many different elements of the Dallas entrepreneurial community spontaneously came together to help Bill. There was no central planning; there were no turf battles. The consistent ingredient was entrepreneurs with a passionate desire to help other entrepreneurs succeed. The difference here is that in Texas, we don't just want each of our own efforts to succeed – we want everybody's entrepreneurial efforts to succeed. That's how entrepreneurship benefits society as a whole and makes it all worthwhile.

What can I say? It's Dallas. That's how we roll.

Get the Boiling Oil Ready

I've blogged before on the topic of computer security and the need for approaches like "asymmetric warfare" to the security problems that our industry – actually our entire society – is experiencing. The recent Target breach is yet another example of how out-of-control the situation is becoming.

I believe we are now on the cusp of a large shift in the corporate and governmental stance on this problem. And this shift may finally begin to turn the tide.

Going On Offense

First you have to understand that as an industry, we've always been in a defensive posture when it comes to cyber attacks. This has been a natural consequence of US law providing no protection for retaliatory responses. Any actions you take against an attacker must not violate the same laws that the attacker violated when they attacked you.

This stance is a purely defensive one – meaning only the US government has the right to retaliate against the hackers, whether that be by legal means or cyber attack. The problem is that the US government doesn't have the resources to effectively track, prosecute and/or retaliate against the hackers. It is not that there are so many hackers; it's that there are so many weak spots for them to attack.

The Internet is like the Wild West where there was one US Marshal for many hundreds of square miles with bands of bandits roaming around. The key difference is back in the Wild West, everybody was armed with weapons to defend themselves. The current state of cyber attacks is that victims get to wear all the body armor they like – but they cannot raise a hand in response.

You cannot win a war if you are always on defense.

An Internet Castle Doctrine

The clear precedent for changing this situation is the concept of self-defense. You can legally take the life of another human being if you do so in defense of your own life. This concept has been around for a very long time and is well tested by and supported by the law and the courts.

In addition to self-defense is the Castle Doctrine. While laws supporting this doctrine do not exist in all states, the concept is pretty simple – the immunity of self-defense is extended to your abode. In other words, your home is treated as your "castle" and you can use lethal force to defend it.

What I believe is needed now is a cyber version of the Castle Doctrine – an "Internet Castle Doctrine". Laws supporting an Internet Castle Doctrine would closely follow the principles of the Castle Doctrine and self-defense. These laws would protect you or your organization if you choose to retaliate against a cyber attack in an offensive fashion.

It seems that most cyber security professionals agree that it is time for this change. Only 30% of IT security leaders were not ready to pursue non-defensive responses to cyber attacks because "too many legal and ethical questions" remain.

Weaponizing Cyber Security

In the same way that the need for self-defense feeds the gun industry, an Internet Castle Doctrine is likely to feed an industry producing cyber self-defense weaponry. CrowdStrike is a startup that came out of stealth mode in 2013 pursuing new approaches to responding to cyber attacks. While they are not offering cyber weaponry yet, they are working on "active defense" systems. There are bound to be more startups quietly working on this too.

Along with the creation of this cyber weapons market will come the inevitable counter arguments that will fuel "cyber weapon control" efforts. As James Lewis, a senior fellow with the Center for Strategic and International Studies put it, enabling counter attacks

Create[s] the risk that some idiot in a company will make a mistake and cause collateral damage that gets us into a war with China.

Yes, collateral damage and unintended consequences are a real concern. However, we have the same concern with guns and self-defense and yet seem to manage well enough. And, at the moment, there doesn't appear to be any other viable alternative to weaponizing and counterattacking.

So, next time the barbarians start attacking your castle walls don't just fill the moat and raise the drawbridge – start thinking about boiling some oil.