Work with private data in a cryptographically secure environment, utilising Intel’s SGX instruction set to improve tenant isolation.
We live in a society where our personal information is increasingly being stored online. It is frequently a single source of truth about us, a location where health information and financial data are saved and maintained, and where choices about what we can and cannot do are made. Critical corporate documents are being maintained online, with paper being phased out for contracts and critical transactions.
But how do we know that data is secure? There’s a certain trust in an encrypted hard drive sitting in a PC under your desk or even in your data center. But what about the cloud? So much of our compute and storage has migrated to services like Azure, either using cloud-native compute or lifted and shifted as virtual infrastructures. Now our data is just one tenant among many in a shared infrastructure where we have no control over how it’s stored and managed.
What is required is a cloud architecture that is supplied as a secure infrastructure for networking, computing, and storage, not only for the code running on it, but protected at such a low level that cloud platform operators cannot access it, even if there is a breach that destroys tenant isolation. It’s a method known as “confidential computing,” in which encryption is used at all levels, including programme execution, which uses the Software Guard Extensions (SGX) to the x64 instruction set, with code executing in trusted execution contexts.
On the computing side of the equation, Azure Private Computing allows you to work with confidential data in a cryptographically secure environment, leveraging Intel’s SGX instruction set to improve tenant isolation. By encrypting memory, information cannot leak between users or between programmes.
When it comes to storing and interacting with stored data, things become more difficult. More than encrypted data is required in this case. We need to know who did what with that information. Consider it an extension of the logs used by current databases, a tool capable of reconstructing every transaction in sequence, replaying it, and arriving at the exact same state. That’s what we mean when we talk about secure ledgers.
Running a secured confidential ledger in Azure
An encrypted log like this is essentially a blockchain, a technology that Microsoft has already experimented with in Azure. However, if you do not require the usage of a blockchain to verify the activities of untrusted parties. You may use a blockchain-based technique to implement the key ledger functions as a stand-alone application that still implements a protected log, without the complexity that come with proof-of-work and proof-of-stake approaches to blockchains.
We’ve seen some of this work in the recently announced Azure SQL secure ledger tables, but now Azure Confidential Ledger takes Microsoft’s ledger technology out of the database, offering it as a simple API that can be used from any application with a simple REST call. Azure Confidential Ledger’s API-based approach goes as far as providing administrative APIs that can be used from your own management tools.
Microsoft describes its approach to ledger technology as “designing ourselves out of the solution.” Only you have access to the ledger, ensuring data integrity that’s not normally provided by cloud solutions. Microsoft’s staff, from its developers to its administrators, are blocked from access to your encrypted data.
Under the hood is a minimal Azure host running a trusted computing base that only supports the ledger and can’t be accessed by other applications, avoiding the risks that come with shared physical memory. Keeping the overall attack surface of the host to a minimum reduces risk, making it harder for a bad actor to compromise your ledger and access its data.
The service has entered public preview (currently with no charge), with a focus on providing an immutable and tamperproof record store. You can set it up from the Azure Portal, via an ARM template, or from the Azure CLI. Access is controlled through certificate-based authentication. Future releases will extend this to Azure Active Directory, adding role-based access control. For now, any code you use will need to work with the Azure identity client.
Other prerequisites include the Confidential Ledger control plane and data plane client libraries. The preview has Python, .NET, and Java libraries, with more promised. Once you’ve installed your chosen set of tools into your development environment, you can either create a new resource group for your ledger or add it to an existing one. Once you’ve opened a resource group, you can register a Confidential Ledger and verify that it’s been created.
Getting started with Azure Confidential Ledger
Once a Confidential Ledger is up and running you can start to write code to use it. One important note: Ledgers need to have globally unique names, so make sure to use one that has a low chance of collision with one from outside your organization.
The two libraries provide different functions. The control plane library controls ledgers, including creating, removing, and displaying them. Before a data plane application may add data to a ledger, all activities must be connected with an Azure account and the fundamental details of the ledger must be set up. Because you will be publishing unstructured data to the ledger, using the data plane library to construct a client is quite straightforward. To authenticate a connection, a client must utilise the ledger certificate in conjunction with its endpoint URL and application credentials. Adding a record is as simple as adding a new entry using a simple string as the entry contents.
Each new entry gets its own unique transaction ID, which can be used to read back data. It’s all very simple, with basic REST API calls that interact with the ledger. You don’t need to worry about the underlying secure execution environment or any of the cryptographic techniques used to store data. The Azure Confidential Ledger provides a sufficiently high-level abstraction from the technology so all that matters is what you write and how you read it back.
A ledger’s function is to store data that is vulnerable to falsification or compromise, safeguarding it against deletion or modification. Using Azure Confidential ledger as part of a line-of-business application helps decrease the risk of fraud by preventing insiders from concealing their actions. It also aids in avoiding some of the consequences of ransomware or other types of assaults. A well-designed ledger can aid in the recovery of lost data in traditional storage facilities. It can, for example, serve as an external storage location for transaction logs or provide an additional layer to a non-relational document store.
The future: confidential computing as a service
Currently the Azure Confidential Ledger is a single-party system, with multiple replicas for redundancy. There are plans to extend it to more than one party, using a similar consortium model as used by the now deprecated Azure Blockchain Service. However, that’s still some ways off, and in practice, much of the benefit of a confidential ledger is to provide a single source of validated truth for a line-of-business system. Ensuring that confidential data is stored securely is perhaps the most important aspect of such a system, especially in regulated industries where significant fines and other penalties can be applied if data is lost in any way.
Tools such as Azure Confidential Ledger allow you to reap the benefits of secure blockchain storage while avoiding the latency and other problems that might arise in large-scale distributed systems. Locking down the system to a set of known safe environments with only API-based access offers another layer of protection, reducing the attack surface. As a result, many of the advantages of secret computing are available with none of the complications. Consider Azure Confidential Ledger to be “confidential computing as a service,” as there is no need to comprehend dealing with SGX instructions, which is something you should expect to see more of in the future.