Deep Dive into Stardust

Making IOTA's Ledger Web3-Ready

TL;DR:
Stardust transforms IOTA into an infrastructure layer for smart contract chains and introduces custom tokens. The new ledger is capable of conditional transfers and NFTs can function as wallets, while additional protocol improvements protect node resources, eliminate client-side trust assumptions and improve load balancing capabilities of the network. Stardust debuts on the new Shimmer network before being ported to the IOTA mainnet.

The IOTA protocol is set to receive its biggest utility upgrade after Chrysalis: Stardust builds on last year's Chrysalis network upgrade and adds unprecedented utility. In this blog post, we explore the origins of Stardust and the detailed benefits it will bring to the table. If you are looking for a condensed introduction, check out Stardust in a nutshell.

Last year's Chrysalis upgrade delivered major improvements to IOTA's performance, stability, reliability, and security while doing away with unnecessarily complex and unorthodox concepts, and instead simplified and aligned with industry standards and best practices. The Chrysalis protocol has been running stable on the IOTA mainnet ever since and protocol efforts have been concentrated on Coordicide with the aim of full-scale decentralization, currently being tested through the IOTA 2.0 DevNet implementation.

Why do we then need Stardust as an interim step between Chrysalis and Coordicide?

In parallel to our work on Coordicide, we have been building the IOTA Smart Contract (ISC) protocol on top of the IOTA 2.0 DevNet. The ISC Beta launched on the DevNet last October and its base layer protocol received support to properly accommodate smart contracts.

We learned a lot from this phase and identified that the needs of ISC on the current IOTA protocol are not fully covered by the decentralization efforts of Coordicide. Hence, it is a logical step to port the new protocol features over to the IOTA mainnet as soon as possible to enable our community to start building smart contract-based use cases on the IOTA mainnet. The Stardust upgrade allows us to enable smart contract support on the mainnet, before Coordicide.

Stardust will debut on Shimmer, IOTA’s staging network, and will undergo thorough testing and community validation. By the time it hits the IOTA mainnet, our community will have had time to learn the ins and outs of the new protocol and build applications and tooling. This way, dApps will be able to launch on the IOTA mainnet together with the new protocol.

Why Stardust?

Naturally, the question then arises: why do we need to modify the current IOTA protocol to enable the running of smart contracts on top of it?

The answer is simple. The second-layer approach employed by ISC enables horizontal scaling and unprecedented levels of throughput but relies on the base IOTA protocol to supply the trustless infrastructure for interoperability.

The keyword here is infrastructure. Think about the Tangle as the interstate highway system: it provides the means for people and goods to flow freely between states, just like the Stardust protocol allows digital assets and information to travel between smart contract chains, each of them functionally equivalent to a blockchain like Ethereum being connected through the IOTA Tangle.

IOTA is well suited to be the foundation for distributed ledger technology (DLT) networks to operate on top of it, because:

The asynchronous nature of the Tangle and the UTXO ledger model allows parallel transaction processing.
The feeless property of the protocol preserves value for inter-chain value transfers.
The UTXO model can be extended with special application logic for trustless atomic operations.

Today, the Chrysalis ledger is optimized for a single application: cryptocurrency payments. Stardust aims to change this and transform IOTA into an infrastructure layer for smart contract chains while at the same time making it a multi-asset ledger. On top of that, the new output types in Stardust will allow you to write arbitrary data directly into the ledger state, which means these datasets are not pruned by mainnet nodes after some time as it is the case with pure data messages on the Tangle.

An illustrative example of that would be the DID document used in IOTA Identity. So far the use-case pattern depended on permanodes to be able to fetch a given DID for verification purposes. With a DID document stored in the IOTA ledger state, it would be available on every single mainnet node for as long as its owner is willing to lock its IOTA storage deposit, which is described in more detail further down in this blog post.

Smart contract requirements

What does an IOTA Smart Contract chain need in order to run on the IOTA infrastructure layer?

A ledger account with a permanent address that is controlled by a rotating committee to send and receive tokens.
A place to store state commitments of the layer 2 (L2) chain.
Custom tokens for cross-chain asset transfers.
Means of facilitating user interaction.

Chrysalis doesn’t support any of these features. The design of the Stardust protocol presents a novel solution to the challenges listed above by extending transaction validation and output unlocking in the layer 1 (L1) ledger with configurable scripts, so-called output types, unlock conditions, and output features.

Let’s explore the solutions in more detail.

A new ledger account

A ledger account is usually an address directly derived from a secret private key. The owner can sign transactions with the secret key and anyone can verify in the ledger whether or not the signature is valid.

However, smart contract chain accounts are controlled by a set of validators that come together to produce a common threshold signature address which can only be unlocked if a threshold number of parties provide their signatures. Every time the validator set is changed, a new common address has to be created.

As a result, the traditional address concept cannot provide a permanent address for smart contract chain accounts. Instead, a new way of thinking about ledger accounts is needed:

What if we could generate and keep track of unique addresses on the protocol level that can rotate the actual private keys used for signing transactions?

For this to work, we need to figure out how to:

Generate unique addresses deterministically.
Persist their state and the controlling keys across transactions.

The first challenge is the easiest. Each transaction in the ledger is unique, therefore we could derive a unique address from the content of the transaction that creates it. Check.

The second challenge requires the modification of the ledger’s transaction validation rules to allow the definition of additional constraints in the context of transactions. A new data structure is also required to store the unique address and the information of the address’ owner. In summary, we have to design a new output type that holds data and, when included in transactions, executes additional validation logic.

The alias output implements this data structure and introduces the alias address as a ledger account. Whenever an alias output is first created in the ledger, a unique address is generated by the protocol and recorded in the output itself, alongside the private key-backed addresses that currently control it. These controllers of the alias account can include the output in transactions to modify it according to the rules defined in its configurable validation script.

For now, it is enough to know that the validation ensures that:

The alias output itself is present on the output side of the transaction. Its state is persisted.
Controller roles can be transferred to any other address in the ledger.
Any funds in the ledger locked under the permanent alias address can be unlocked by proving ownership of the alias output itself; that is, by proving that one is the controller and is able to unlock the output.

Storing state commitments

Smart contract chains are like blockchains that produce state updates in blocks. Each block identifies the state of the blockchain corresponding to that specific block. By anchoring the L2 state on L1, we mean recording a cryptographic commitment to the current L2 blockchain state inside the alias output. This is beneficial to trustlessly synchronize the L2 ledger state with the L1 ledger state and opens up the door for zero-knowledge L2 state update verification on L1 in the future.

Stardust enables storing arbitrary data in outputs via a feature called metadata feature. Smart contract chains are therefore able to store their state commitments directly in their ledger account.

However, the question of data storage in outputs shines the light on a more general problem: the size of the ledger. The bigger the ledger size, the bigger the load put on the physical hardware that runs network nodes. We all know that there is no such thing as infinite resources in the physical world. Therefore, a system is needed to regulate data storage and hence ledger size. Enter storage deposits.

Storage deposit mechanism

Storage deposits protect the network from excessive dust. The term “dust protection” comes from the way the ledger size can be inflated by distributing funds into ever smaller chunks (called “dust”) that force all network nodes to store more and more accounts holding just a tiny amount of tokens. As an example: at the time of writing, for the price of $100 an attacker could create 100,000,000 addresses that contain one IOTA each. Every entry in the network’s ledger requires a small amount of data to be managed by nodes. Therefore, if not properly mitigated, dust can become a significant problem in networks that do not require transaction fees.

Chrysalis already implemented a solution for this problem, though not without caveats. The database bloat problem is less transparent in Chrysalis because ledger entries are fixed and small in size. Furthermore, it is not possible to store user-defined data in ledger accounts.

Stardust introduces a new solution that limits the amount of data to be stored in ledger accounts in relation to the amount of funds (deposits) they hold. By holding funds, one rents storage space in the ledger. It is important to realize that storage must be regulated based on a scarce resource; otherwise, the ledger size could grow indefinitely.

Also note that, by just having a ledger account, permanent data storage is already forced on all network nodes. Therefore, we require an account to have a minimum amount of funds to be created. The very same principles are present in Bitcoin and Cardano, but database bloat attacks are less likely to manifest in them due to the discouraging factor of transaction fees.

Custom tokens

As soon as tokens could be deployed on smart contract platforms, the number of new tokens (such as ERC-20 or ERC-721) in the crypto space grew exponentially. They are a vital part of the dApp ecosystem and help projects to raise capital, implement novel governance structures or provide access to unique features.

ISC supports the Ethereum virtual machine (EVM) and the creation of such tokens on L2. But if custom tokens are created on Layer 2, how can such tokens be moved between different smart contract chains? Other networks usually require a bridge to be implemented in order to translate between two chains, but these bridges are highly centralized, are high-value targets, and consequently often hacked, as demonstrated in the latest Ronin bridge hack.

Stardust implements the IOTA Tokenization framework, which is designed to be a protocol built-in, trustless and atomic bridge solution for cross-chain asset transfers between ISC chains. In short: With Stardust, there is no need to build any complicated bridges. The IOTA Tangle, to which all chains are connected, is the bridge.

At the core of the framework is the transformation of the IOTA ledger into a multi-asset ledger. Anyone, including smart contract chains, can create supply-controlled custom tokens directly on the L1 Tangle. This is the basis for asset wrapping, a mechanism that allows the creation of L1 representations of assets originating from smart contract chains. Once wrapped and represented on the IOTA base layer, assets can be sent feelessly to another address on the IOTA ledger, including addresses controlled by other L2 chains.

Native tokens

Fungible tokens are implemented in the Stardust protocol via native tokens. Such tokens are minted into existence in token foundries. The ledger account controlling the foundry is referred to as the issuer. They have the right to melt tokens in their possession, decreasing the total supply, while holders can burn tokens (which does not remove them from the total supply but from the circulating supply).

Transacting with native tokens is done via regular transfers and all possible output types support holding them, whether they are smart contract chain accounts or regular addresses. Due to the storage deposit mechanism described above, a transfer must always include the base currency of the network to cover the network storage, which makes pure native token transfers slightly more complicated. Rest assured that all requirements will be abstracted from end-users through graphical user interfaces like Firefly. We will explore the requirements for native asset transfers later in this blog post.

The initial version of Stardust will launch with support for the Simple Token Scheme. Token schemes define the supply control rules, that is, what token issuers are allowed to do. The simple scheme puts an upper limit on the total supply of tokens, guaranteeing that there is no way for issuers to inflate the supply more than the maximum supply defined upon token creation.

Non-fungible tokens

In contrast to the previously-described fungible custom tokens, non-fungible tokens (NFTs) are unique tokens in the ledger that carry with them immutable metadata. They are implemented in Stardust as a standalone output type, called NFT output. The concept described previously for a permanent smart contract address account fits very well here, therefore the NFT output is actually a cousin of the alias output but with additional perks.

NFTs in Stardust:

Receive a globally unique identifier upon minting that also functions as a permanent address controlled by the current owner.
May have immutably attached data set upon minting, which is carried by the NFT for its lifetime,
May also have immutably attached issuer identity set upon minting.
Can own other NFTs, tokens, or any assets both on the base layer and on smart contract chains. Each NFT functions as a standalone wallet with a permanent address.
Can be burnt by their owners; for example, to free up tokens locked in them that may be required as storage deposit.

Minting an NFT is as inexpensive as sending a regular transfer. In fact, it is feeless! However, since NFTs may carry additional data with them, issuers have to supply enough tokens to the NFT output to cover the storage deposit. When the NFT is burned, this deposit is fully refunded to its current owner.

User interaction with smart contracts

So far the features for setting up smart contract chains, storing state commitments, and cross-chain asset transfers have been discussed. How do regular users interact with chains?

ISC defines two types of interaction with smart contract chains:

On-ledger requests originating from the base protocol,
Off-ledger requests sent directly to the L2 chain.

As the name suggests, off-ledger requests are sent directly to the L2 smart contract chain, therefore they are of no interest to the Stardust protocol. On-ledger requests on the other hand require special support from the base protocol, implemented in Stardust via two new outputs, Unlock Conditions and Output Features.

Output unlock conditions

An on-ledger request is an output of a transaction that is sent to the address of the smart contract chain on L1. As such, the output should only be unlockable by the smart contract chain for the request to be processed.

In Stardust, this feature is supported by the Address Unlock Condition, which is merely a generalization of the output locking concept in Chrysalis and extended with the new address types.

An on-ledger request may deposit native assets into the chain (native tokens or NFTs). As discussed before, sending native tokens without involving the base currency is not possible, and it is a rather harsh demand for users to always deposit base tokens together with native tokens into chains. Therefore, a new Storage Deposit Return Unlock Condition makes it possible to automatically reclaim the storage deposit of outputs by the sender.

If Alice wishes to send exactly one AliceCoin to Bob, she would create an output on Bob’s address with Storage Deposit Amount of IOTA Tokens + 1 AliceCoin and define a return condition that she awaits Storage Deposit Amount of IOTA Tokens back. Bob can only consume this output in a transaction if he refunds Alice with at least that amount of IOTAs.

What if Bob never consumes the output? Alice’s storage deposit is locked forever! That’s why Alice is smart enough to define an Expiration Unlock Condition on the output as well. She can set an expiration deadline until which Bob has to consume the output, otherwise its ownership reverts exclusively to Alice.

This works well for on-ledger requests too: one can specify that the smart contract chain can only take the native asset from the output, but must return the storage deposit. With expiration, the user makes sure that the request is either processed by the chain within the deadline or is canceled altogether. It is a unique feature of Stardust and the ISC system.

Another handy feature is the Timelock Unlock Condition that prevents the output from being unlocked until the timelock deadline is met. Why is it so important? Because you can precisely time smart contract requests with it. As soon as the deadline is over, the smart contract chain picks up and executes the request.

These unlock conditions are currently supported by Stardust because they are required for smart contracts, but many ideas are floating around the Tangle Improvement Proposal (TIP) repository discussion forum, swapping being one of them.

Output features

Features that don’t affect the actual unlocking of the output are defined as Output Features. For smart contracts, one of the most important features has already been mentioned: the Metadata Feature. This one is also essential for users and on-ledger requests. The Metadata Feature of the output holds the actual call data, that is, the instructions to be carried out by smart contracts.

When Alice wants to invoke a smart contract on Chain A from her IOTA account:

She prepares an IOTA transaction in which she creates an output (on-ledger request) with funds to the permanent address of Chain A.
In the Metadata Feature of the output, she encodes the call data: “call entry point X of smart contract Y with parameters A, B, and C”
As soon as the transaction confirms on IOTA, Chain A recognizes that there is a new on-ledger request sitting on its address.
Chain A picks up the request and executes the call encoded in the Metadata Feature, advancing the L2 state of the blockchain and settling the result on IOTA via a state update anchoring transaction.

How does the chain know that Alice created the output – that is, that the request should be carried out under Alice’s name? It doesn’t have access to the IOTA transaction, only to its output. And even if it had, what if Bob and Alice sent requests in the same transaction?

The Stardust solution is the Sender Feature. Outputs can explicitly specify their senders, while transaction validation ensures that the sender address is unlocked on the input side of the transaction. Therefore, the owner of the sender address must have agreed to carry out the on-ledger request under their name.

Note that on-ledger requests can be sent from any IOTA address, including from NFT Addresses and Alias Addresses. The latter makes it possible for Chain A to execute a request on Chain B, taking smart contract composability to the next level: cross-chain composability. The former, sending smart contract requests from NFT Addresses, introduces new possibilities for dApp developers as specific smart contract functionality can be made available to NFT holders only.

All of the unlock conditions and features are supported by Basic Outputs, which are the primary vehicle for on-ledger requests, as well as by NFT Outputs.

Additional protocol improvements

So far, we’ve presented the origin story of most Stardust protocol features. But besides catering to smart contract requirements, there are more improvements compared to Chrysalis.

Replay protection

Transactions in DLT are signed instructions that modify the ledger state. The existence of parallel networks with the same ledger functionality (think about Shimmer and IOTA) opens up the possibility of replay attacks, which is when an attacker reissues a valid signed transaction in another network.

To prevent even the slightest chance of this happening, Stardust introduces a network identifier field in the content of the signed transactions. As a result, even if a transaction is otherwise correct, a different network than the intended would immediately reject it based on the mismatching network identifier.

Inputs commitment

In a UTXO-based ledger, transactions reference the inputs they wish to consume by their unique identifiers. Clients gather the content of inputs by accessing the inputs' identifiers in nodes. If you don’t run your own node, your wallet probably talks to an explorer or indexer application that in turn fetches data from a network node. Do you fully trust a third party to give you the correct content of the inputs owned by you to construct a transaction? What if their infrastructure is hacked?

Luckily, with Stardust, you don’t need to trust third parties. Transactions include an Inputs Commitment field that (as the name suggests) is a cryptographic commitment to the content of the inputs of the transaction. If for any reason your wallet was supplied with malicious data and constructed a transaction based on that data, the network will realize that there is a mismatch between what the network thinks your inputs are and what your wallet does.

This mechanism protects not only users but also smart contract chains. An attacker might be able to eclipse the connection of L2 validators to L1 nodes and start replacing request content to steal user funds, but with this security mechanism, such malicious transactions would be rejected by the base protocol, causing the chain to realize it was eclipsed because it didn’t produce a valid state update.

Off-loading data processing from the core protocol

IOTA is unique in that it provides data-only transactions in the Tangle. However, use-cases that build on this feature face two major problems:

The Tangle is permissionless, therefore anyone can send data messages with any content and the messages are not authenticated with signatures like value transactions. The source of the data published through the Tangle is not identifiable by the core protocol.
Data use-case applications often rely on structured, filtered and processed application-specific data. Such functionality puts unnecessary load on network nodes running the core protocol.

An example of the latter is the data indexation feature present in the Chrysalis upgrade. Data use-case application clients want to be able to fetch data from the Tangle that is of interest to them, therefore data must be indexed to support client queries. As described above, Chrysalis does not prevent anyone from spamming any indices with any data, possibly breaking client functionality if they do not implement any mitigation.

Stardust removes any data processing from the core protocol, as supporting use-case specific processing requirements in the core protocol is unfeasible – and anyway, it would jeopardize node performance and hence transaction throughput in the network.

Data in Stardust is published via Tagged Data Payloads, which are treated as binary data by the protocol. It is recommended that the processing and exposure of application-specific data published through these payloads are implemented by second-layer protocols. One major benefit of this approach is its flexibility: each application can define and implement its own requirements – for example, to authenticate data payloads based on digital signatures, indexing by custom fields or validating payloads against expected data structures.

The re-engineered node software provides a Remote Procedure Call (gRPC) interface, called IOTA Node Extension (INX) to external applications to interact with nodes for example to listen to all network activity. Data use-cases are encouraged to build their custom data processing applications and connect them to the Tangle via INX.

To stay consistent with this new architecture, Stardust also removes the ledger indexing from the core protocol and implements a ledger indexer application via an INX module.

Dynamic Proof of Work

Proof of Work (PoW) is currently employed in IOTA for congestion control. Every transaction must include a small amount of computational work if it is to be considered valid. Note that, while in blockchain networks miners compete to solve the cryptographic puzzle of PoW first and waste a huge amount of energy in the process, IOTA users who submit transactions to the network take part in a cooperative effort.

Chrysalis has a fixed PoW difficulty factor for a unit of data submitted to the network. Therefore, the actual complexity of the challenge for a transaction is dependent on its length only.

Stardust protocol design incorporates a dynamic PoW difficulty factor based on the congestion of the network. The added utility of the protocol upgrade could result in higher network activity. If this load reaches a certain threshold near the limit of the network throughput capabilities, the protocol self-adjusts the PoW difficulty factor. When the load is reduced, the process reverses to lower the difficulty until the threshold is reached again.

This mechanism will be supported by the network after the first-ever fluid protocol upgrade, meaning the feature will be activated in the already running, live network without any downtime. The node software is being refactored to handle many more of such future protocol upgrades.

Summary

This concludes our list of new protocol features in Stardust. Let’s recap what we have learned so far.

Stardust was born as the smart contract support module of the IOTA 2.0 protocol. Initial testing of ISC chains on the IOTA 2.0 DevNet yielded a lot of learnings and improvements for the prototype. Stardust was identified as a standalone protocol module that can already be ported to the IOTA mainnet as a huge step in the direction of IOTA 2.0 in a production setting.

Shimmer will launch with the Stardust protocol to undergo thorough testing and validation before it debuts on the mainnet. This also gives enough time for community projects to build and test their dApps before their eventual launch on IOTA.

To add unprecedented levels of utility to the IOTA ledger, Stardust introduces:

New types of ledger accounts.
Several new output types.
Output unlock conditions and features.
Feeless native tokens and NFTs.
Mechanisms to off-load data processing.

It’s not only smart contract chains that will be able to leverage the new feature: so will regular users and future innovators.

Additionally, a basket of protocol improvements and architecture changes aims to improve security, performance and network behavior in times of heavy congestion.

What’s next?

The Stardust protocol specifications have been thoroughly reviewed and implementation is well underway. The node software prototype is undergoing its internal alpha testing, while client and tooling development is ongoing at full steam. Crucial parts of this effort are the wallet library support for the new ledger models and concepts, as well as the new user interaction flows of the Firefly wallet.

In parallel, the ISC team is working day and night to refactor several key components of the ISC node software for Stardust. Under development is a new ISC virtual machine called Stardust VM, which is fully compatible with new base protocol features.

Stardust will enter public beta testing once developer tooling is implemented and node software alpha testing gives satisfactory results. The main audience for public beta testing are developers and, most importantly, community projects.

Shimmer will launch with Stardust and a complete set of user-facing tools to let non-technical people also experience the magic of the added utility of the new protocol upgrade. Eventually, Stardust will find its way to the IOTA mainnet together with ISC and smart contracts to accelerate Web3 innovation in the IOTA ecosystem. Sounds exciting? Embark on this journey with us and follow the Stardust development on GitHub, contribute to a Tangle Improvement Proposal, share your own ideas, or engage with the community on Discord to discover more about the possibilities of Stardust.