Cookin' with Substrate ๐Ÿด๐Ÿ˜‹๐Ÿด

Substrate Recipes is a collection of simple code patterns that demonstrate best practices when building blockchains with Substrate.

This book is built with mdbook and deployed via github pages. The code used to build this book is open source and open for contributions.

Thanks

Thanks to the following for suggestions and content contribution: gautamdhameja, joshorndorff, joepetrowski, ltfschoen, nczhu, shawntabrizi

License

This work is licensed under both the Creative Commons Share Attribution-ShareAlike 4.0 International License and the MIT License.

Happy Coding :)

Getting Started

If you're here, you probably already know about Substrate and are ready to start building. If not, I recommend visiting the official documentation. For a high level overview, read these blog posts:

How to Use This Book

Start by cloning the repo on github:

git clone https://github.com/substrate-developer-hub/recipes

As you read through the book, practice compiling and testing recipes in recipes/kitchen. You can't learn how to code by reading about it -- play with the code in the kitchen, extract patterns, and apply them to a problem that you want to solve!

You can read the book chronologically or jump around. Personally, I prefer jumping around, but people learn in different ways :)

Regardless of the approach you take, it is useful to recognize that coding is all about abstraction. To accelerate your progress, I recommend skimming the patterns in this book, composing them into interesting projects, and abstracting your own unique recipes. Feel free to reach out for guidance on Stack Overflow or in the Substrate Technical Riot channel.

Chef's Choice

My favorite recipes include

Notable Substrate Tutorials and Projects

Substrate Developer Hub

Before anything else, I'd recommend starting with the Substrate Collectables tutorial; it'll help you hit the ground running with an interactive sample project.

To learn more about how to build novel blockchains with Substrate, check out the utxo-workshop.

If interested in token-based mechanisms, look no further than the Substrate TCR. The full tutorial covers Substrate best practices.

If interested in smart contracts on Substrate, check out the substrate-contracts-workshop.

Prerequisites

If you do not have substrate installed on your machine, run:

curl https://getsubstrate.io -sSf | bash

Substrate Templates

Substrate package contains the UI, module, and runtime templates for building with Substrate. The substrate-module-template is the simplest path to experimenting with Substrate. Modules are modular pieces of code that can be composed within a single runtime.

Likewise, the substrate-node-template provides all necessary scaffolding for running a functional Substrate node. Each Substrate runtime contains multiple modules that comprise the logic of the defined Substrate blockchain.

The substrate-ui provides a template for building a compatible UI that works with the node template.

Runtime Module

Clone the substrate-module-template

$ git clone https://github.com/shawntabrizi/substrate-module-template

build with

$ cargo build

test with

$ cargo test

Runtime Node

Clone the substrate-node-template and add module logic to runtime/src/template.rs.

Update the runtime root lib.rs file to include the new Event<T> type under the module's Trait implementation


# #![allow(unused_variables)]
#fn main() {
/// in root `lib.rs`
mod mymodule;

impl mymodule::Trait for Runtime {
    type Event = Event<T>;
}
#}

Include the Event<T> type in the module's definition in the construct_runtime macro block.


# #![allow(unused_variables)]
#fn main() {
/// in root `lib.rs`
construct_runtime!(
    pub enum Runtime for Log(InteralLog: DigestItem<Hash, Ed25519AuthorityId) where
        Block = Block,
        NodeBlock = opaque::Block,
        InherentData = BasicInherentData
    {
        ...
        MyModule: mymodule::{Module, Call, Storage, Event<T>},
    }
);
#}

Updating the Runtime

Compile runtime binaries

cd runtime
cargo build --release

Delete the old chain before you start the new one (this is a very useful command sequence when building and testing runtimes!)

./target/release/substrate-example purge-chain --dev
./target/release/substrate-example --dev

Event

In Substrate, transaction finality does not guarantee the execution of functions dependent on the given transaction. To verify that functions have executed successfully, emit an event at the bottom of the function body.

Events notify the off-chain world of successful state transitions

To declare an event, use the decl_event macro.

Recipes

More Resources

Adding Machine

A simple adding machine which checks for overflow and emits an event with the result, without using storage.

In the module file


# #![allow(unused_variables)]
#fn main() {
pub trait Trait: system::Trait {
    type Event: From<Event> + Into<<Self as system::Trait>::Event>;
}

decl_module! {
    pub struct Module<T: Trait> for enum Call where origin: T::Origin {
        fn deposit_event() = default;

        fn add(_origin, val1: u32, val2: u32) -> Result {
            // checks for overflow
            let result = match val1.checked_add(val2) {
                Some(r) => r,
                None => return Err("Addition overflowed"),
            };
            Self::deposit_event(Event::Added(val1, val2, result));
            Ok(())
        }
    }
}

decl_event!(
    pub enum Event {
        Added(u32, u32, u32),
    }
);
#}

Incrementing Balances

This recipe demonstrates how we can store a balance type and increment it using a runtime method. The first step is to add srml-balances to the Cargo.toml file like so

[dependencies.balances]
default_features = false
git = 'https://github.com/paritytech/substrate.git'
package = 'srml-balances'
branch = 'v1.0'

Don't forget to add balances/std under the [features] section:

[features]
default = ['std']
std = [
    'parity-codec/std',
    'support/std',
    'system/std',
    'runtime-primitives/std',
    'balances/std',
]

The decl_event macro generates an Event type which needs to be exposed in the module. This type inherits the balances trait. See single value storage recipe for more information on Substrate specific types


# #![allow(unused_variables)]
#fn main() {
/// in module file
pub trait Trait: balances::Trait {
    type Event: From<Event<Self>> to Into<<Self as system::Trait>::Event>;
}
#}

Our stored balance type is kept in the decl_storage block


# #![allow(unused_variables)]
#fn main() {
decl_storage! {
    trait Store for Module<T: Trait> as IncBalance {
        BalanceVal get(balance_val): Option<T::Balance>;
    }
}
#}

The NewBalance event associated with updating BalanceVal uses the generic type B = <T as balances::Trait>::Balance


# #![allow(unused_variables)]
#fn main() {
/// in module file
decl_event!(
    pub enum Event<T> where B = <T as balances::Trait>::Balance {
        NewBalance(B),
    }
);
#}

To use events in the runtime, it is necessary to add a function to deposit the declared events. Within the decl_module block, add a new function


# #![allow(unused_variables)]
#fn main() {
/// in module file, decl_module block
fn deposit_event<T>() = default();
#}

Note: If your event uses only Rust primitive types, then the generic <T> is unncesssary and can be omitted. See adding machine for an example of this

After checking for the successful state transition in the body of a function, the corresponding event should be invoked.


# #![allow(unused_variables)]
#fn main() {
/// in module file, decl_module block
pub fn accumulate_dummy(origin, increase_by: T::Balance) -> Result {
    // This is a public call, so we ensure that the origin is some signed account.
    let _sender = ensure_signed(origin)?;

    // use the `::get` on the storage item type itself
    let balance_val = <BalanceVal<T>>::get();

    // Calculate the new value.
    let new_balance = balance_val.map_or(increase_by, |val| val + increase_by);

    // Put the new value into storage.
    <BalanceVal<T>>::put(new_balance);

    // Deposit an event to let the outside world know this happened.
    Self::deposit_event(RawEvent::NewBalance(increase_by));

    // All good.
    Ok(())
}
#}

Permissioned Function with Generic Event

This recipe contains a permissioned function which can only be called by the Owner. An event is emitted when the function is successfully executed.

The imports are similar to previous event recipes with the additional import of the support::StorageValue.


# #![allow(unused_variables)]
#fn main() {
// other imports
use support::{StorageValue};
#}

In the decl_storage block, designate the AccountId of the owner that can invoke the permissioned function.


# #![allow(unused_variables)]
#fn main() {
decl_storage! {
    trait Store for Module<T: Trait> as RuntimeExampleStorage {
        Owner get(owner): T::AccountId;
    }
}
#}

When this AccountId is changed, it is useful to emit an event to notify any relevant actors off-chain.


# #![allow(unused_variables)]
#fn main() {
decl_event!(
    pub enum Event<T> where AccountId = <T as system::Trait>::AccountId {
        OwnershipTransferred(AccountId, AccountId),
    }
);
#}

The main logic is contained in the runtime methods. Our first runtime method initiates the ownership. Before doing so, it verifies that no current owner exists.


# #![allow(unused_variables)]
#fn main() {
/// in decl_module
fn init_ownership(origin) -> Result {
    ensure!(!<Owner<T>>::exists(), "Owner already exists");
    let sender = ensure_signed(origin)?;
    <Owner<T>>::put(&sender);
    Self::deposit_event(RawEvent::OwnershipTransferred(sender.clone(), sender));
    Ok(())
}
#}

The second runtime method transfers ownership. Before doing so, it checks that the invocation is made by the current owner.


# #![allow(unused_variables)]
#fn main() {
fn transfer_ownership(origin, newOwner: T::AccountId) -> Result {
    let sender = ensure_signed(origin)?;
    ensure!(sender == Self::owner(), "This function can only be called by the owner");
    <Owner<T>>::put(&newOwner);
    Self::deposit_event(RawEvent::OwnershipTransferred(sender, newOwner));
    Ok(())
}
#}

This recipe can be extended to create permissioned functions that limit invocations to members of specified groups.

Storage

Use the decl_storage macro to define type-safe, persistent data that needs to be stored on-chain.

For cryptocurrencies, storage might consist of a mapping between account keys and corresponding balances.

More generally, blockchains provide an interface to store and interact with data in a verifiable and globally irreversible way. In this context, data is stored in a series of snapshots, each of which may be accessed at a later point in time, but, once created, snapshots are considered irreversible.

Generally speaking, you may store arbitrary data, as long as its data type is serializable in Substrate i.e. implements Encode and Decode traits.

Recipes

More Resources

Single Value

Substrate supports all primitive Rust types (bool, u8, u32, etc) as well as some custom types specific to Substrate (Hash, Balance, BlockNumber, etc).

Basic Storage

Within a specific module, a single value (u32 type) is stored in the runtime with this syntax:


# #![allow(unused_variables)]
#fn main() {
decl_storage! {
    trait Store for Module<T: Trait> as Example {
        MyValue: u32;
    }
}
#}

Storage Interaction

To interact with single storage values, it is necessary to import the support::StorageValue type. Functions used to access a StorageValue are defined in srml/support:


# #![allow(unused_variables)]
#fn main() {
/// Get the storage key.
fn key() -> &'static [u8];

/// true if the value is defined in storage.
fn exists<S: Storage>(storage: &S) -> bool {
    storage.exists(Self::key())
}

/// Load the value from the provided storage instance.
fn get<S: Storage>(storage: &S) -> Self::Query;

/// Take a value from storage, removing it afterwards.
fn take<S: Storage>(storage: &S) -> Self::Query;

/// Store a value under this key into the provided storage instance.
fn put<S: Storage>(val: &T, storage: &S) {
    storage.put(Self::key(), val)
}

/// Mutate this value
fn mutate<R, F: FnOnce(&mut Self::Query) -> R, S: Storage>(f: F, storage: &S) -> R;

/// Clear the storage value.
fn kill<S: Storage>(storage: &S) {
    storage.kill(Self::key())
}
#}

Therefore, the syntax to "put" Value:


# #![allow(unused_variables)]
#fn main() {
<MyValue<T>>::put(1738);
#}

and to "get" Value:


# #![allow(unused_variables)]
#fn main() {
let my_val = <MyValue<T>>::get();
#}

Getter Syntax

Storage values can also be declared with a get function to provide cleaner syntax for getting values.


# #![allow(unused_variables)]
#fn main() {
decl_storage! {
    trait Store for Module<T: Trait> as Example {
        MyValue get(value_getter): u32;
    }
}
#}

The get parameter is optional, but, by including it, the module exposes a getter function (fn value_getter() -> u32).

To "get" the Value with the getter function


# #![allow(unused_variables)]
#fn main() {
let my_val = Self::value_getter();
#}

Setter Syntax

Here is an example of a module that stores a u32 value in runtime storage and provides a function set_value to set the given u32. This code follows convention for naming setters in Rust.


# #![allow(unused_variables)]
#fn main() {
use srml_support::{StorageValue, dispatch::Result};

pub trait Trait: system::Trait {}

decl_module! {
    pub struct Module<T: Trait> for enum Call where origin: T::Origin {
        fn set_value(origin, value: u32) -> Result {
            // check sender signature to verify permissions
            let sender = ensure_signed(origin)?; 
            <MyValue<T>>::put(value);
            Ok(())
        }
    }
}

decl_storage! {
    trait Store for Module<T: Trait> as Example {
        MyValue: u32;
    }
}
#}

Substrate Specific Types

To access Substrate specific types, the module's Trait must inherit from the SRML. For example, to access the Substrate types Hash, AccountId, and BlockNumber, it is sufficient to inherit the system module:


# #![allow(unused_variables)]
#fn main() {
pub trait Trait: system::Trait {}
#}

This provides access to the types Hash, AccountId, and BlockNumber anywhere that specifies the generic <T: Trait> using T::<Type>. It also provides access to other useful types, declared in the pub Trait {} block in systems/src/lib.rs.

If access to the Balances type is required, then the trait bound should specify


# #![allow(unused_variables)]
#fn main() {
pub trait Trait: balances::Trait {}
#}

Because the balances module also inherits the system module, it grants access to all the types in system as well as those specified in the pub Trait {} block in balances/src/lib.rs. For a practical example of this syntax, see the Incrementing Balance event recipe.

Mapping Accounts to Balances

Mappings are a very powerful primitive. A stateful cryptocurrency might store a mapping between accounts and balances (see token example). Likewise, mappings prove useful when representing owned data. By tracking ownership with maps, it is easy manage permissions for modifying values specific to individual users or groups.

Within a specific module, a key-value mapping (between u32 types) can be stored with this syntax:


# #![allow(unused_variables)]
#fn main() {
decl_storage! {
    trait Store for Module<T: Trait> as Example {
        MyMap: map u32 => u32;
    }
}
#}

Basic Map Usage

To interact with a storage map, it is necessary to import the support::StorageMap type. Functions used to access a StorageValue are defined in srml/support:


# #![allow(unused_variables)]
#fn main() {
/// Get the prefix key in storage.
fn prefix() -> &'static [u8];

/// Get the storage key used to fetch a value corresponding to a specific key.
fn key_for(x: &K) -> Vec<u8>;

/// true if the value is defined in storage.
fn exists<S: Storage>(key: &K, storage: &S) -> bool {
    storage.exists(&Self::key_for(key)[..])
}

/// Load the value associated with the given key from the map.
fn get<S: Storage>(key: &K, storage: &S) -> Self::Query;

/// Take the value under a key.
fn take<S: Storage>(key: &K, storage: &S) -> Self::Query;

/// Store a value to be associated with the given key from the map.
fn insert<S: Storage>(key: &K, val: &V, storage: &S) {
    storage.put(&Self::key_for(key)[..], val);
}

/// Remove the value under a key.
fn remove<S: Storage>(key: &K, storage: &S) {
    storage.kill(&Self::key_for(key)[..]);
}

/// Mutate the value under a key.
fn mutate<R, F: FnOnce(&mut Self::Query) -> R, S: Storage>(key: &K, f: F, storage: &S) -> R;
#}

To insert a (key, value) pair into a StorageMap named MyMap:


# #![allow(unused_variables)]
#fn main() {
<MyMap<T>>::insert(key, value);
#}

To query MyMap for the value corresponding to a key:


# #![allow(unused_variables)]
#fn main() {
let value = <MyMap<T>>::get(key);
#}

Simple Token Transfer

To implement a simple token transfer with Substrate,

  1. set total supply
  2. establish ownership upon configuration of circulating tokens
  3. coordinate token transfers with the runtime functions

# #![allow(unused_variables)]
#fn main() {
decl_storage! {
  trait Store for Module<T: Trait> as TokenTransfer {
    pub TotalSupply get(total_supply): u64 = 21000000; // (1)

    pub GetBalance get(get_balance): map T::AccountId => u64; // (3)

    Init get(is_init): bool; // (2)
  }
}
#}

Declare an event for when token transfers occur to notify clients


# #![allow(unused_variables)]
#fn main() {
decl_event!(
    pub enum Event<T> where AccountId = <T as system::Trait>::AccountId {
        // notifies upon token transfers
        Transfer(AccountId, AccountId, u64), // (from, to, value)
    }
);
#}

Define the business logic in runtime methods


# #![allow(unused_variables)]
#fn main() {
decl_module! {
  pub struct Module<T: Trait> for enum Call where origin: T::Origin {
    // initialize the default event for this module
    fn deposit_event<T>() = default;

    // initialize the token
    // transfers the total_supply amout to the caller
    fn init(origin) -> Result {
      let sender = ensure_signed(origin)?;
      ensure!(Self::is_init() == false, "Already initialized.");

      <GetBalance<T>>::insert(sender, Self::total_supply());

      <Init<T>>::put(true);

      Ok(())
    }

    // transfer tokens from one account to another
    fn transfer(_origin, to: T::AccountId, value: u64) -> Result {
      let sender = ensure_signed(_origin)?;
      let sender_balance = Self::get_balance(sender.clone());
      ensure!(sender_balance >= value, "Not enough balance.");

      let updated_from_balance = sender_balance.checked_sub(value).ok_or("overflow in calculating balance")?;
      let receiver_balance = Self::get_balance(to.clone());
      let updated_to_balance = receiver_balance.checked_add(value).ok_or("overflow in calculating balance")?;
      
      // reduce sender's balance
      <GetBalance<T>>::insert(sender.clone(), updated_from_balance);

      // increase receiver's balance
      <GetBalance<T>>::insert(to.clone(), updated_to_balance);

      Self::deposit_event(RawEvent::Transfer(sender, to, value));
      
      Ok(())
    }
  }
}
#}

S/O gautamdhameja/substrate-demo for providing this recipe!

Implementing Lists with Maps

Substrate does not natively support a list type since it may encourage dangerous habits. Unless explicitly guarded against, a list will add unbounded O(n) complexity to an operation that will only charge O(1) fees (Big O notation refresher). This opens an economic attack vector on your chain.

Emulate a list with a mapping and a counter like so:


# #![allow(unused_variables)]
#fn main() {
use support::{StorageValue, StorageMap};

decl_storage! {
    trait Store for Module<T: Trait> as Example {
        TheList get(the_list): map u32 => T::AccountId;
        TheCounter get(the_counter): u32;
    }
}
#}

This code allows us to store a list of participants in the runtime represented by AccountIds. Of course, this implementation leaves many unanswered questions such as

  • How to add and remove elements?
  • How to maintain order under mutating operations?
  • How to verify that an element exists before removing/mutating it?

This recipe answers those questions with snippets from relevant code samples:

Adding/Removing Elements in an Unbounded List

If the size of the list is not relevant, the implementation is straightforward. Note how it is still necessary to verify the existence of elements in the map before attempting access.

To add an AccountId, increment the the_count and insert an AccountId at that index:


# #![allow(unused_variables)]
#fn main() {
// decl_module block
fn add_member(origin) -> Result {
    let who = ensure_signed(origin)?;

    // increment the counter
    <TheCounter<T>>::mutate(|count| *count + 1);

    // add member at the largest_index
    let largest_index = <TheCounter<T>>::get();
    <TheList<T>>::insert(largest_index, who.clone());

    Self::deposit_event(RawEvent::MemberAdded(who));

    Ok(())
} 
#}

To remove an AccountId, call the remove method for the StorageMap type at the relevant index. In this case, it isn't necessary to update the indices of other proposals; order is not relevant.


# #![allow(unused_variables)]
#fn main() {
// decl_module block
fn remove_member_unbounded(origin, index: u32) -> Result {
    let who = ensure_signed(origin)?;

    // verify existence
    ensure!(<TheList<T>>::exists(index), "an element doesn't exist at this index");
    let removed_member = <TheList<T>>::get(index);
    <TheList<T>>::remove(index);

    Self::deposit_event(RawEvent::MemberRemoved(removed_member));

    Ok(())
}
#}

Because the code doesn't update the indices of other AccountIds in the map, it is necessary to verify an AccountId's existence before removing it, mutating it, or performing any other operation.

Swap and Pop for Ordered Lists

To preserve storage so that the list doesn't continue growing even after removing elements, invoke the swap and pop algorithm:

  1. swap the element to be removed with the element at the head of the list (the element with the highest index in the map)
  2. remove the element recently placed at the highest index
  3. decrement the TheCount value.

Use the swap and pop algorithm to remove elements from the list.


# #![allow(unused_variables)]
#fn main() {
// decl_module block
fn remove_member_ordered(origin, index: u32) -> Result {
    let who = ensure_signed(origin)?;

    ensure!(<TheList<T>>::exists(index), "an element doesn't exist at this index");

    let largest_index = <TheCounter<T>>::get();
    let member_to_remove = <TheList<T>>::take(index);
    // swap
    if index != largest_index {
    let temp = <TheList<T>>::take(largest_index);
    <TheList<T>>::insert(index, temp);
    <TheList<T>>::insert(largest_index, member_to_remove.clone());
    }
    // pop
    <TheList<T>>::remove(largest_index);
    <TheCounter<T>>::mutate(|count| *count - 1);

    Self::deposit_event(RawEvent::MemberRemoved(member_to_remove.clone()));

    Ok(())
}
#}

Keep the same logic for inserting proposals (increment TheCount and insert the entry at the head of the list)

Linked Map

To trade performance for relatively simple code, utilize the linked_map data structure. By implementing EnumarableStorageMap in addition to StorageMap, linked_map provides a method head which yields the head of the list, thereby making it unnecessary to also store the LargestIndex. The enumerate method also returns an Iterator ordered according to when (key, value) pairs were inserted into the map.

To use linked_map, import EnumerableStorageMap. Here is the new declaration in the decl_storage block:


# #![allow(unused_variables)]
#fn main() {
use support::{StorageMap, EnumerableStorageMap}; // no StorageValue necessary

decl_storage! {
    trait Store for Module<T: Trait> as Example {
        LinkedList get(linked_list): linked_map u32 => T::AccountId;
        LinkedCounter get(linked_counter): u32;
    }
}
#}

The add_member_linked method is logically equivalent to the previous add method. Here is the new remove_member_linked method:


# #![allow(unused_variables)]
#fn main() {
// decl_module block
fn remove_member_linked(origin, index: u32) -> Result {
    let who = ensure_signed(origin)?;

    ensure!(<LinkedList<T>>::exists(index), "A member does not exist at this index");

    let head_index = <LinkedList<T>>::head().unwrap();
    let member_to_remove = <LinkedList<T>>::take(index);
    let head_member = <LinkedList<T>>::get(head_index);
    <LinkedList<T>>::insert(index, head_member);
    <LinkedList<T>>::remove(head_index);

    Ok(())
}
#}

The only caveat is that this implementation incurs some performance costs (vs solely using StorageMap and StorageValue) because linked_map heap allocates the entire map as an iterator in order to implement the enumerate method.

Generic Structs

In Rust, a struct, or structure, is a custom a custom data type that lets you name and package together multiple related values that make up a meaningful group. If youโ€™re familiar with an object-oriented language, a struct is like an objectโ€™s data attributes (read more in The Rust Book).

To define a custom struct for the runtime, the following syntax may be used:


# #![allow(unused_variables)]
#fn main() {
#[derive(Encode, Decode, Default, Clone, PartialEq)]
pub struct MyStruct<A, B> {
    some_number: u32,
    some_generic: A,
    some_other_generic: B,
}
#}

In the code snippet above, the derive macro is declared to ensure MyStruct conforms to shared behavior according to the specified traits: Encode, Decode, Default, Clone, PartialEq

To use the Encode and Decode traits, it is necessary to import them from support::codec:


# #![allow(unused_variables)]
#fn main() {
use support::codec::{Encode, Decode};
#}

By storing types in MyStruct as generics, it is possible to access custom Substrate types like AccountId, Balance, and Hash.

For example, to store a mapping from AccountId to MyStruct with some_generic as the Balance type and some_other_generic as the Hash type:


# #![allow(unused_variables)]
#fn main() {
decl_storage! {
    trait Store for Module<T: Trait> as Example {
        MyMap: map T::AccountId => MyStruct<T::Balance, T::Hash>;
    }
}
#}

Basic Interaction

To push values and modify the map


# #![allow(unused_variables)]
#fn main() {
decl_module! {
    pub struct Module<T: Trait> for enum Call where origin: T::Origin {
        fn create_struct(origin, number: u32, balance: T::Balance, secret: T::Hash) -> Result {
            let sender = ensure_signed(origin)?;

            let new_struct = MyStruct {
                some_number: number,
                some_generic: balance,
                some_other_generic: secret,
            };

            <MyMap<T>>::insert(sender, new_struct);
            Ok(())
        }
    }
}
#}

Nested Structs

This basic runtime shows how to store custom, nested structs using a combination of Rust primitive types and Substrate specific types via generics.


# #![allow(unused_variables)]
#fn main() {
pub trait Trait: balances::Trait {}

#[derive(Encode, Decode, Default)]
pub struct Thing <Hash, Balance> {
    my_num: u32,
    my_hash: Hash,
    my_balance: Balance,
}

#[derive(Encode, Decode, Default)]
pub struct SuperThing <Hash, Balance> {
    my_super_num: u32,
    my_thing: Thing<Hash, Balance>,
}

decl_module! {
    pub struct Module<T: Trait> for enum Call where origin: T::Origin {
        fn set_mapping(_origin, key: u32, num: u32, hash: T::Hash, balance: T::Balance) -> Result {
            let thing = Thing { 
                            my_num: num, 
                            my_hash: hash, 
                            my_balance: balance
                        };
            <Value<T>>::insert(key, thing);
            Ok(())
        }

        fn set_super_mapping(_origin, key: u32, super_num: u32, thing_key: u32) -> Result {
            let thing = Self::value(thing_key);
            let super_thing = SuperThing { 
                            my_super_num: super_num, 
                            my_thing: thing
                        };
            <SuperValue<T>>::insert(key, super_thing);
            Ok(())
        }
    }
}

decl_storage! {
    trait Store for Module<T: Trait> as RuntimeExampleStorage {
        Value get(value): map u32 => Thing<T::Hash, T::Balance>;
        SuperValue get(super_value): map u32 => SuperThing<T::Hash, T::Balance>;
    }
}
#}

For more information, see the Substrate TCR and the full tutorial

UI Interaction

To access the value of the struct via the User Interface (UI), it is necessary to import the structure of the new type such that the UI understand how to decode it. See Cryptokitties Collectables Tutorial for directions on how to configure accordingly with Polkadot UI or Substrate UI.

Higher Order Arrays with Tuples and Maps

naive social network recipe below below

To represent ownership of multiple items across multiple users, tuples can be used alongside maps in order to emulate arrays.

For example, consider a scenario in which persistent storage keeps track of a social network graph in which each user (represented by an AccountId) has a list of other friends. In this case, it would be convenient to use a 2 dimensional array like


# #![allow(unused_variables)]
#fn main() {
SocialNetwork[AccountId][Index] -> AccountId
#}

With this data structure, check how many friends a given AccountId has by calling


# #![allow(unused_variables)]
#fn main() {
SocialNetwork[AccountId].length()
#}

To emulate this data structure in the context of the Substrate runtime storage, use tuples and maps (declared in a decl_storage!{} block like previous examples):


# #![allow(unused_variables)]
#fn main() {
MyFriend get(my_friend): map (T::AccountId, u32) => T::AccountId;
FriendsCount get(friends_count): map T::AccountId => u32;
#}

Patterns that use mappings to emulate higher order data structures are common when managing runtime storage on Substrate.

Naive Social Network

We can use this pattern to manage whitelists and blacklists. This is especially useful in the context of social networks for adding/removing friends and blocking unfriendly participants.

The relevant state transitions are encoded in the decl_event block


# #![allow(unused_variables)]
#fn main() {
decl_event!(
    pub enum Event<T> where AccountId = <T as system::Trait>::AccountId {
      NewFriend(AccountId),
      FriendRemoved(AccountId),
      Blocked(AccountId),
      UnBlocked(AccountId),
    }
);
#}

Our storage items contain a higher order array represented by the items described previously.


# #![allow(unused_variables)]
#fn main() {
decl_storage! {
  trait Store for Module<T: Trait> as SocialNetwork {
    MyFriend get(my_friend): map (T::AccountId, u32) => T::AccountId;
    FriendsCount get(friends_count): map T::AccountId => u32;
    AllFriends get(all_friends): map T::AccountId => Vec<T::AccountId>;
    Blocked get(blocked): map T::AccountId => Vec<T::AccountId>;
  }
}
#}

We also include two vectors for a user's friends and the participants that they have blocked. These vectors are convenient when paired with runtime methods for verifying membership.


# #![allow(unused_variables)]
#fn main() {
impl<T: Trait> Module<T> {
  pub fn friend_exists(current: T::AccountId, friend: T::AccountId) -> bool {
    // search for friend in AllFriends vector
    <AllFriends<T>>::get(current).iter()
          .any(|&ref a| a == &friend)
  }

  pub fn is_blocked(current: T::AccountId, other_user: T::AccountId) -> bool {
    // search for friend in Blocked vector
    <Blocked<T>>::get(current).iter()
          .any(|&ref a| a == &other_user)
  }
}
#}

By returning bool, we can easily use these methods in ensure! statements to verify relevant state conditions before making requests in the main runtime methods. For example, in the remove_friend runtime method, we need to ensure that the friend to be removed is an existing friend.


# #![allow(unused_variables)]
#fn main() {
ensure!(Self::friend_exists(user.clone(), old_friend.clone()), "old friend is not a friend");
#}

Similarly, when we block a user, we should check that the user isn't already blocked.


# #![allow(unused_variables)]
#fn main() {
ensure!(!Self::is_blocked(user.clone(), blocked_user.clone()), "user is already blocked");
#}

The full logic for this sample can be found in the kitchen in storage/arrays.

To see another example of how to use tuplies to emulate higher order arrays, see the Substrate Collectables Tutorial.

NOTE: DoubleMap is a map with two keys; this storage item may also be useful for implementing higher order arrays

Module Menu

The official Substrate documentation provides a comprehensive overview of the Substrate runtime module libraries. Although modules are designed to be stand-alone, the modules in the Substrate Runtime Module Library provide useful code patterns that are applicable to many applications leveraging the framework.

Unlike in smart contract development, the way to emulate these patterns is not to directly utilize these modules. Instead, the best approach either implements the same logic in the new context or utilizes a trait from srml/support to guide the new implementation. By abstracting shared behavior from the runtime modules into srml/support, Substrate makes it easy to extract and enforce best practices in the unique runtime. You can find the trait documentation here.

Module Tour

  • Aura - manages offline reporting for Aura consensus
  • Balances - handles accounts and balances
  • Consensus - manages the authority set for the native code
  • Contracts - functionality for the runtime to deploy and execute WebAssembly smart contracts
  • Council - handles voting and maintenance of council members
  • Democracy - handles administration of general stakeholder voting
  • Executive - dispatches incoming extrinsic calls to the respective modules in the runtime
  • Grandpa - manages the GRANDPA authority set ready for the native code
  • Indices - an index is a short form of an address; this module handles allocation of indices for a newly created accounts
  • Session - allows validators to manage their session keys, provides a function for changing the session length, and handles session rotation
  • Staking - manage funds at stake by network maintainers
  • Sudo - allows a single account to execute dispatchable functions
  • System - low-level access to core types and cross-cutting utilities
  • Timestamp - get and set the on-chain time
  • Treasury - keeps account of currency in a pot and manages the subsequent deployment of these funds

Safety and Optimization

Unlike conventional software development kits that abstract away low-level decisions, Substrate grants developers fine-grain control over the underlying implementation. This approach fosters high-performance, modular applications. At the same time, it also demands increased attention from developers. To quote the late Uncle Ben, with great power comes great responsibility.

Indeed, Substrate developers have to exercise incredible caution. The bare-metal control that they maintain over the runtime logic introduces new attack vectors. In the context of blockchains, the cost of bugs scale with the amount of capital secured by the application. Likewise, developers should generally abide by a few rules when building with Substrate. These rules may not hold in every situation; Substrate offers optimization in context.

Testing

Testing is not (yet) covered in the Substrate Recipes, but there is a great introduction to testing in the context of Substrate in the Crypto Collectables Tutorial. I also have enjoyed the following articles/papers on testing that apply to code organization more generally:

Module Development Criteria

  1. Modules should be independent pieces of code; if your module is tied to many other modules, it should be a smart contract. See the substrate-contracts-workshop for more details with respect to smart contract programming on Substrate.

  2. It should not be possible for your code to panic after storage changes. Poor error handling in Substrate can brick the blockchain, rendering it useless thereafter. With this in mind, it is very important to structure code according to declarative, condition-oriented design patterns. See more in the declarative programming section.

Declarative Programming

Within each runtime module function, it is important to perform all checks prior to any storage changes. When coding on most smart contract platforms, the stakes are lower because panics on contract calls will revert any storage changes. Conversely, Substrate requires greater attention to detail because mid-function panics will persist any prior changes made to storage.

Using the Ensure Macro

Substrate developers should use ensure! checks at the top of each runtime function's logic to verify that all of the requisite checks pass before performing any storage changes. Note that this is similar to require() checks at the top of function bodies in Solidity contracts.

The Social Network recipe demonstrated how we can create separate runtime methods to verify necessary conditions in the main methods.


# #![allow(unused_variables)]
#fn main() {
impl<T: Trait> Module<T> {
  pub fn friend_exists(current: T::AccountId, friend: T::AccountId) -> bool {
    // search for friend in AllFriends vector
    <AllFriends<T>>::get(current).iter()
          .any(|&ref a| a == &friend)
  }

  pub fn is_blocked(current: T::AccountId, other_user: T::AccountId) -> bool {
    // search for friend in Blocked vector
    <Blocked<T>>::get(current).iter()
          .any(|&ref a| a == &other_user)
  }
}
#}

"By returning bool, we can easily use these methods in ensure! statements to verify relevant state conditions before making requests in the main runtime methods."


# #![allow(unused_variables)]
#fn main() {
// in the remove_friend method
ensure!(Self::friend_exists(user.clone(), old_friend.clone()), "old friend is not a friend");

...
// in the block method
ensure!(!Self::is_blocked(user.clone(), blocked_user.clone()), "user is already blocked");
#}

Indeed, this pattern of extracting runtime checks into separate functions and invoking the ensure macro in their place is useful. It produces readable code and encourages targeted testing to more easily identify the source of logic errors.

For a deeper dive into the "Verify First, Write Last" pattern, see the relevant section in the Substrate Collectables tutorial as well as Substrate Best Practices. This github comment is also very useful for visualizing the declarative pattern in practice.

Bonus Reading

Verifying Signed Messages

It is often useful to designate some functions as permissioned and, therefore, accessible only to a defined group. In this case, we check that the transaction that invokes the runtime function is signed before verifying that the signature corresponds to a member of the permissioned set.


# #![allow(unused_variables)]
#fn main() {
let who = ensure_signed(origin)?;
ensure!(Self::is_member(&who), "user is not a member of the group");
#}

We can define is_member similar to the helper methods in the Social Network recipe by defining a vector of AccountIds (current_member) that contains all members. We then search this vector for the AccountId in question within the body of the is_member method.


# #![allow(unused_variables)]
#fn main() {
impl<T: Trait> Module<T> {
    pub fn is_member(who: &T::AccountId) -> bool {
        Self::current_member().iter()
            .any(|&ref a| a == who)
    }
}
#}

To read more about checking for signed messages, see the relevant section in the Substrate collectables tutorial.

Checking for Collisions

Often times we may intend for keys to be unique identifiers that map to a specific storage item. In this case, it is necessary to check for collisions before adding new entries.

For example, it is common to use the hash of an object as the unique identifier in a map defined in the decl_storage block. Before adding a new value to the map, check that the key (hash) doesn't already have an associated value in the map. If it does, it is necessary to decide between the new item and the existing item to prevent an inadvertent key collision. In most cases, the new value is rejected.


# #![allow(unused_variables)]
#fn main() {
fn insert_value(origin, hash: Hash, value: u32) {
    // check that key doesn't have an associated value
    ensure!( !(Self::map::exists(&hash)), "key already has an associated value" );

    // add key-value pair
    <Map<T>>::insert(hash, value);
}
#}

See how the Substrate Collectables Tutorial covers this pattern.

Optimization Tricks

Runtime overhead in Substrate corresponds to the efficiency of the underlying Rust code. Therefore, it is essential to use clean, efficient Rust patterns for performance releases. This section introduces common approaches for optimizing Rust code in general and links to resources that may guide further investigation.

This section was inspired by and pulls heavily from

Premature Optimization

Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. - Page 268 of Structured Programming with goto Statements by Donald Knuth

Before worrying about performance optimizations, focus on optimizing for readability, simplicity, and maintainability. The first step when building anything is achieving basic functionality. Only after establishing a minimal viable sample is it appropriate to consider performance-based enhancements. With that said, severe inefficiency does open attack vectors for Substrate runtimes (see the next section). Moreover, the tradeoff between optimization and simplicity is not always so clear...

A common misconception is that optimized code is necessarily more complicated, and that therefore optimization always represents a trade-off. However, in practice, better factored code often runs faster and uses less memory as well. In this regard, optimization is closely related to refactoring, since in both cases we are paying into the code so that we may draw back out again later if we need to. - src

Rust API Guidelines

Also, use clippy!

Efficiency => Security in Substrate

We call an algorithm efficient if its running time is polynomial in the size of the input, and highly efficient if its running time is linear in the size of the input. It is important for all on-chain algorithms to be highly efficient, because they must scale linearly as the size of the Polkadot network grows. In contrast, off-chain algorithms are only required to be efficient. - Web3 Research

See Substrate Best Practices for more details on how efficiency influences the runtime's economic security.

Related Reading

Rust Zero-Cost Abstractions

Substrate developers should take advantage of Rust's zero cost abstractions.

Articles

Tweets

Video

Entering unsafe Waters ๐Ÿดโ€โ˜ ๏ธ

Please read The Rustonomicon before experimenting with the dark magic that is unsafe

To access an element in a specific position, use the get() method. This method performs a double bound check.


# #![allow(unused_variables)]
#fn main() {
for arr in array_of_arrays {
    if let Some(elem) = arr.iter().get(1738) {
        println!("{}", elem);
    }
}
#}

The .get() call performs two checks:

  1. checks that the index will return Some(elem) or None
  2. checks that the returned element is of type Some or None

If bound checking has already been performed independently of the call, we can invoke .getunchecked() to access the element. Although this is unsafe to use, it is equivalent to C/C++ indexing, thereby improving performance when we already know the element's location.


# #![allow(unused_variables)]
#fn main() {
for arr in array_of_arrays {
    println!("{}", unsafe { arr.get_unchecked(1738) })
}
#}

NOTE: if we don't verify the input to .getunchecked(), the caller may access whatever is stored in the location even if it is a memory address outside the slice

Fearless Concurrency && Asynchrony

As a systems programming language, Rust provides significant flexibility with respect to low-level optimizations. Specifically, Rust provides fine-grain control over how you perform computation, delegate said computation to the OS's threads, and schedule state transitions within a given thread. There isn't space in this book to go into significant detail, but I'll try to provide resources/reading that have helped me get up to speed. For a high-level overview, Stjepan Glavina provides the following descriptions in Lock-free Rust: Crossbeam in 2019:

  • Rayon splits your data into distinct pieces, gives each piece to a thread to do some kind of computation on it, and finally aggregates results. Its goal is to distribute CPU-intensive tasks onto a thread pool.
  • Tokio runs tasks which sometimes need to be paused in order to wait for asynchronous events. Handling tons of such tasks is no problem. Its goal is to distribute IO-intensive tasks onto a thread pool.
  • Crossbeam is all about low-level concurrency: atomics, concurrent data structures, synchronization primitives. Same idea as the std::sync module, but bigger. Its goal is to provide tools on top of which libraries like Rayon and Tokio can be built.

To dive deeper down these ๐Ÿฐ holes

Asynchrony

Are we async yet?

Conceptual

Projects

Concurrency

Conceptual

Projects

Dessert ๐Ÿซ

Check out awesome-substrate for projects, events, and all the latest Substrate news!

Featured Tutorials