Substrate Recipes 🍴😋🍴

Substrate Recipes is a collection of simple code patterns that demonstrate best practices when building blockchains with Substrate. The repo used to build this book is open source. Check out the contributions guidelines for an overview of the structure and directions for getting involved.

The current scope is limited to module development and runtime configuration. To learn more about Substrate, see the official documentation.

What is Substrate?

Substrate is a framework for building blockchains. For a high level overview, read the following blog posts:

How to Use This Book

Start by cloning the repo on github:

git clone https://github.com/substrate-developer-hub/recipes

As you read through the book, practice compiling and testing recipes in recipes/kitchen. You can't learn to code by reading about it -- play with the code in the kitchen, extract patterns, and apply them to a problem that you want to solve!

It is useful to recognize that coding is all about abstraction. To accelerate your progress, I recommend skimming the patterns in this book, composing them into interesting projects, and building your own recipes.

Reach out for guidance on Stack Overflow or in the Substrate Technical Riot channel.

Learn Rust

To be productive with substrate requires some familiarity with Rust. Fortunately, the Rust community is known for comprehensive documentation and tutorials. The most common resource for initially learning Rust is The Rust Book. To see examples of popular crate usage patterns, Rust by Example is also convenient.

API Design

To become more familiar with commmon design patterns in Rust, the following links might be helpful:

Optimizations

To optimize runtime performance, Substrate developers should make use of iterators, traits, and Rust's other "zero cost abstractions":

It is not (immediately) necessary to become familiar with multithreading because the runtime operates in a single-threaded context. Even so, an optimized substrate node architecture will use a custom RPC interface. Moreover, the runtime might take advantage of the offchain workers API to minimize the computation executed on-chain. Effectively using these features requires increased familiarity with advanced Rust.

For a high-level overview of concurrency in Rust, Stjepan Glavina provides the following descriptions in Lock-free Rust: Crossbeam in 2019:

  • Rayon splits your data into distinct pieces, gives each piece to a thread to do some kind of computation on it, and finally aggregates results. Its goal is to distribute CPU-intensive tasks onto a thread pool.
  • Tokio runs tasks which sometimes need to be paused in order to wait for asynchronous events. Handling tons of such tasks is no problem. Its goal is to distribute IO-intensive tasks onto a thread pool.
  • Crossbeam is all about low-level concurrency: atomics, concurrent data structures, synchronization primitives. Same idea as the std::sync module, but bigger. Its goal is to provide tools on top of which libraries like Rayon and Tokio can be built.

Asynchrony

Are we async yet?

Conceptual

Projects

Concurrency

Conceptual

Projects

Getting Started

If you do not have a Substrate development environment setup on your machine, please install it by following these directions.

For Linux / macOS

# Setup Rust and Substrate
curl https://getsubstrate.io -sSf | bash

For Windows

Refer to our Substrate Installation on Windows.

Kitchen Overview

The recipes/kitchen folder contains all the code necessary to run a Substrate node. Let us call it the Kitchen Node. There are three folders inside:

  • node - contains the code to start the Kitchen Node.
  • runtimes - contains the runtime of the Kitchen Node.
  • modules - each runtime includes multiple modules. Each module gives the runtime a new set of functionality. Most of the recipe module code we discuss afterwards is stored under this folder.

This section teaches users to interact with recipes/kitchen by

Run the Kitchen Node

To run the code in the recipes, git clone the source repository. We also want to kick-start the node compilation as it may take about 30 minutes to complete depending on your hardware.

git clone https://github.com/substrate-developer-hub/recipes.git
cd recipes/kitchen/node
./scripts/init.sh

# This step takes a while to complete
cargo build --release

Notes

Refer to the following sections to:

Once the compilation is completed, you can first purge any existing blockchain data (useful to start your node from a clean state in future) and then start the node.

# Inside `recipes/kitchen/node` folder

# Purge any existing blockchain data. Enter `y` upon prompt.
./target/release/kitchen-node purge-chain --dev

# Start the Kitchen Node
./target/release/kitchen-node --dev

Interact with the Kitchen Node

If you followed the instructions to get your node running, you should see blocks created on the console. You can now use our Polkadot-JS Apps to interact with your locally running node. You will use the Chain state tab to query the blockchain status and Extrinsics to send transactions to the blockchain.

To configure relevant type definitions, follow the directions in the polkadot-js docs.

Using the Kitchen

Module Fundamentals

kitchen/modules/hello-substrate

Clone the substrate module template:

git clone https://github.com/substrate-developer-hub/substrate-module-template

At the top of the src/lib.rs file, import the following from srml-support:


# #![allow(unused_variables)]
#fn main() {
use support::{decl_module, decl_event, decl_storage, StorageValue, StorageMap};
use system::ensure_signed;
#}

The blockchain's runtime storage is configured in decl_storage.


# #![allow(unused_variables)]
#fn main() {
decl_storage! {
    trait Store for Module<T: Trait> as HelloWorld {
        pub LastValue get(fn last_value): u64;
        pub UserValue get(fn user_value): map T::AccountId => u64;
    }
}
#}

Defined in decl_module, the runtime methods specify acceptable interaction with runtime storage.


# #![allow(unused_variables)]
#fn main() {
decl_module! {
    pub struct Module<T: Trait> for enum Call where origin: T::Origin {
        fn deposit_event() = default;

        pub fn set_value(origin, value: u64) {
            let sender = ensure_signed(origin)?;
            LastValue::put(value);
            UserValue::<T>::insert(&sender, value);
            Self::deposit_event(RawEvent::ValueSet(sender, value));
        }
    }
}
#}

Events are declared in decl_event. The emission of events is used to determine successful execution of the logic in the body of runtime methods.


# #![allow(unused_variables)]
#fn main() {
decl_event!{
    pub enum Event<T> where
        AccountId = <T as system::Trait>::AccountId,
    {
        ValueSet(AccountId, u64),
    }
}
#}

It is also possible to declare an error type for runtime modules with decl_error

Event

kitchen/modules/simple-event, kitchen/modules/generic-event

In Substrate, transaction finality does not guarantee the execution of functions dependent on the given transaction. To verify that functions have executed successfully, emit an event at the bottom of the function body.

Events notify the off-chain world of successful state transitions

To declare an event, use the decl_event macro.

Simple Event

The simplest example of an event uses the following syntax


# #![allow(unused_variables)]
#fn main() {
decl_event!(
    pub enum Event {
        EmitInput(u32),
    }
);
#}

The event is emitted at the bottom of the do_something function body:


# #![allow(unused_variables)]
#fn main() {
Self::deposit_event(Event::EmitInput(new_number));
#}

Events with Module Types

Sometimes events might emit types from the module Trait. When the event uses types from the module, it is necessary to specify additional syntax


# #![allow(unused_variables)]
#fn main() {
decl_event!(
    pub enum Event<T> where AccountId = <T as system::Trait>::AccountId {
        EmitInput(AccountId, u32),
    }
);
#}

The syntax for deposit_event now takes the RawEvent type because it is generic over the module Trait


# #![allow(unused_variables)]
#fn main() {
Self::deposit_event(RawEvent::EmitInput(user, new_number));
#}

See the next example to use the simple event syntax in the context of verifying successful execution of an adding machine

Module Development Criteria

  1. Modules should be relatively independent pieces of code; if your module is tied to many other modules, it should be a smart contract. See the substrate-contracts-workshop for more details with respect to smart contract programming on Substrate. Also, use traits for abstracting shared behavior.

  2. It should not be possible for your code to panic after storage changes. Poor error handling in Substrate can brick the blockchain, rendering it useless thereafter. With this in mind, it is very important to structure code according to declarative, condition-oriented design patterns. See more in the declarative programming section.

ensure!

Within each runtime module function, it is important to perform requisite checks prior to any storage changes. Unlike existing smart contract platforms, Substrate requires greater attention to detail because mid-function panics will persist any prior changes made to storage.

Place ensure! checks at the top of each runtime function's logic to verify that all requisite conditions are met before performing any storage changes. Note that this is similar to require() checks at the top of function bodies in Solidity contracts.

In the set storage and iteration, a vector was stored in the runtime to allow for simple membership checks for methods only available to members.


# #![allow(unused_variables)]
#fn main() {
decl_storage! {
    trait Store for Module<T: Trait> as VecMap {
        Members get(fn members): Vec<T::AccountId>;
    }
}
...
impl<T: Trait> Module<T> {
    fn is_member(who: &T::AccountId) -> bool {
        <Members<T>>::get().contains(who)
    }
}
#}

"By returning bool, we can easily use these methods in ensure! statements to verify relevant state conditions before making requests in the main runtime methods."


# #![allow(unused_variables)]
#fn main() {
fn member_action(origin) -> Result {
    let member = ensure_signed(origin)?;
    ensure!(Self::is_member(&member), "not a member => cannot do action");
    // <action && || storage change>
    Ok(())
}
#}

Indeed, this pattern of extracting runtime checks into separate functions and invoking the ensure macro in their place is useful. It produces readable code and encourages targeted testing to more easily identify the source of logic errors.

This github comment might help when visualizing declarative patterns in practice. TODO: Ellaborate github comment as it is no longer accessible.

Bonus Reading

Fixed Point Arithmetic

in progress

Computers are imprecise with large numbers. Indeed, floating point arithmetic is not used in the runtime because it can introduce nondeterministic calculations.

Permill, Perbill, Fixed64...

Checking for Overflows/Underflows

We can use the checked traits in substrate-primitives to protect against overflow/underflow when incrementing/decrementing objects in our runtime. To follow the Substrat collectable tutorial example, use checked_add() to safely handle the possibility of overflow when incremementing a global counter. Note that this check is similar to SafeMath in Solidity.


# #![allow(unused_variables)]
#fn main() {
use runtime_primitives::traits::CheckedAdd;

let all_people_count = Self::num_of_people();

let new_all_people_count = all_people_count.checked_add(1).ok_or("Overflow adding a new person")?;
#}

ok_or() transforms an Option from Some(value) to Ok(value) or None to Err(error). The ? operator facilitates error propagation. In this case, using ok_or() is the same as writing


# #![allow(unused_variables)]
#fn main() {
let new_all_people_count = match all_people_count.checked_add(1) {
    Some (c) => c,
    None => return Err("Overflow adding a new person"),
};
#}

Verifying Signed Messages

It is often useful to designate some functions as permissioned and, therefore, accessible only to a defined group. In this case, we check that the transaction that invokes the runtime function is signed before verifying that the signature corresponds to a member of the permissioned set.


# #![allow(unused_variables)]
#fn main() {
let who = ensure_signed(origin)?;
ensure!(Self::is_member(&who), "user is not a member of the group");
#}

We can define is_member similar to the helper methods in the Social Network recipe by defining a vector of AccountIds (current_member) that contains all members. We then search this vector for the AccountId in question within the body of the is_member method.


# #![allow(unused_variables)]
#fn main() {
impl<T: Trait> Module<T> {
    pub fn is_member(who: &T::AccountId) -> bool {
        Self::current_member().iter()
            .any(|&ref a| a == who)
    }
}
#}

To read more about checking for signed messages, see the relevant section in the Substrate collectables tutorial.

Custom Origin

  • todo

Runtime Configuration

The runtime gives context to module development. Indeed, modules are libraries that expose runtime methods to adjust runtime storage. These libraries are configured in the runtime with explicit assignment of the types declared in the module Trait.

To see how the srml modules are configured, see the node/runtime

smpl-treasury

kitchen/modules/treasury

This recipe demonstrates how srml-treasury instantiates a pot of funds and schedules funding.

Instantiate a Pot

To instantiate a pool of funds, import ModuleId and AccountIdConversion from sr-primitives.


# #![allow(unused_variables)]
#fn main() {
use runtime_primitives::{ModuleId, traits::AccountIdConversion};
#}

With these imports, a MODULE_ID constant can be generated as an identifier for the pool of funds. This identifier can be converted into an AccountId with the into_account() method provided by the AccountIdConversion trait.


# #![allow(unused_variables)]
#fn main() {
const MODULE_ID: ModuleId = ModuleId(*b"example ");

impl<T: Trait> Module<T> {
    pub fn account_id() -> T::AccountId {
        MODULE_ID.into_account()
    }

    fn pot() -> BalanceOf<T> {
        T::Currency::free_balance(&Self::account_id())
    }
}
#}

Accessing the pot's balance is as simple as using the Currency trait to access the balance of the associated AccountId.

Proxy Transfers

In srml/treasury, approved spending proposals are queued in runtime storage before they are scheduled for execution. For the example dispatch queue, each entry represents a request to transfer BalanceOf<T> to T::AccountId from the pot.


# #![allow(unused_variables)]
#fn main() {
decl_storage! {
    trait Store for Module<T: Trait> as STreasury {
        /// the amount, the address to which it is sent
        SpendQ get(fn spend_q): Vec<(T::AccountId, BalanceOf<T>)>;
    }
}
#}

In other words, the dispatch queue holds the AccountId of the recipient (destination) in the first field of the tuple and the BalanceOf<T> in the second field. The runtime method for adding a spend request to the queue looks like this


# #![allow(unused_variables)]
#fn main() {
decl_module! {
    pub struct Module<T: Trait> for enum Call where origin: T::Origin {
        // uses the example treasury as a proxy for transferring funds
        fn proxy_transfer(origin, dest: T::AccountId, amount: BalanceOf<T>) -> Result {
            let sender = ensure_signed(origin)?;

            let _ = T::Currency::transfer(&sender, &Self::account_id(), amount)?;
            <SpendQ<T>>::mutate(|requests| requests.push((dest.clone(), amount)));
            Self::deposit_event(RawEvent::ProxyTransfer(dest, amount));
            Ok(())
        }
    }
}
#}

This method transfers some funds to the pot along with the request to transfer the same funds from the pot to a recipient (the input field dest: T::AccountId).

NOTE: Instead of relying on direct requests, srml/treasury coordinates spending decisions through a proposal process.

Scheduling Spending

To schedule spending like srml/treasury, first add a configurable module constant in the Trait. This constant determines how often the spending queue is executed.


# #![allow(unused_variables)]
#fn main() {
pub trait Trait: system::Trait {
    /// Period between successive spends.
    type SpendPeriod: Get<Self::BlockNumber>;
}
#}

This constant is invoked in the runtime method on_finalize to schedule spending every T::SpendPeriod::get() blocks.


# #![allow(unused_variables)]
#fn main() {
decl_module! {
    pub struct Module<T: Trait> for enum Call where origin: T::Origin {
        // other runtime methods
        fn on_finalize(n: T::BlockNumber) {
            if (n % T::SpendPeriod::get()).is_zero() {
                Self::spend_funds();
            }
        }
    }
}
#}

To see the logic within spend_funds, see the kitchen/treasury. This recipe could be extended to give priority to certain spend requests or set a cap on the spends for a given spend_funds() call.

Mapping Accounts to Balances

Mappings are a very powerful primitive. A stateful cryptocurrency might store a mapping between accounts and balances. Likewise, mappings prove useful when representing owned data. By tracking ownership with maps, it is easy manage permissions for modifying values specific to individual users or groups.

To implement a simple token transfer with Substrate,

  1. set total supply
  2. establish ownership upon configuration of circulating tokens
  3. coordinate token transfers with the runtime functions

# #![allow(unused_variables)]
#fn main() {
decl_storage! {
  trait Store for Module<T: Trait> as TokenTransfer {
    pub TotalSupply get(fn total_supply): u64 = 21000000; // (1)

    pub GetBalance get(fn get_balance): map T::AccountId => u64; // (3)

    Init get(fn is_init): bool; // (2)
  }
}
#}

Declare an event for when token transfers occur to notify clients


# #![allow(unused_variables)]
#fn main() {
decl_event!(
    pub enum Event<T> where AccountId = <T as system::Trait>::AccountId {
        // notifies upon token transfers
        Transfer(AccountId, AccountId, u64), // (from, to, value)
    }
);
#}

Define the business logic in runtime methods


# #![allow(unused_variables)]
#fn main() {
decl_module! {
  pub struct Module<T: Trait> for enum Call where origin: T::Origin {
    // initialize the default event for this module
    fn deposit_event() = default;

    // initialize the token
    // transfers the total_supply amout to the caller
    fn init(origin) -> Result {
      let sender = ensure_signed(origin)?;
      ensure!(Self::is_init() == false, "Already initialized.");

      <GetBalance<T>>::insert(sender, Self::total_supply());

      <Init<T>>::put(true);

      Ok(())
    }

    // transfer tokens from one account to another
    fn transfer(_origin, to: T::AccountId, value: u64) -> Result {
      let sender = ensure_signed(_origin)?;
      let sender_balance = Self::get_balance(sender.clone());
      ensure!(sender_balance >= value, "Not enough balance.");

      let updated_from_balance = sender_balance.checked_sub(value).ok_or("overflow in calculating balance")?;
      let receiver_balance = Self::get_balance(to.clone());
      let updated_to_balance = receiver_balance.checked_add(value).ok_or("overflow in calculating balance")?;

      // reduce sender's balance
      <GetBalance<T>>::insert(sender.clone(), updated_from_balance);

      // increase receiver's balance
      <GetBalance<T>>::insert(to.clone(), updated_to_balance);

      Self::deposit_event(RawEvent::Transfer(sender, to, value));

      Ok(())
    }
  }
}
#}

S/O gautamdhameja/substrate-demo for providing this recipe!

see the TCR tutorial for an extension in the form of a token curated registry

Dessert 🍫

Check out awesome-substrate for projects, events, and all the latest Substrate news!

Featured Tutorials