Philippe Charrière

Posted on Aug 1, 2022 • Updated on Aug 27, 2022

Extend Wasm with host functions thanks to Wazero

#wasm #wasi #tinygo #golang

Wazero is a very new WebAssembly runtime written in Go (with zero dependencies). This project comes with a set of very instructive examples. By reading these examples, I discovered and understood two things:

How to create Host Functions
How to "exchange" strings between the WebAssembly module and the host (as you know, WebAssembly only understands numbers).

The Host Functions are functions defined inside the host program (for example, the Go program that will "use" Wazero, load and run the Wasm module). From the Wasm module perspective, a host function can be imported and called by the Wasm module.

In other words: according to the WASI specification, a wasm module is still minimal. For example, by default, it cannot display information in a terminal (fmt.Println("hello") does not work). But we can create a print(something) function in the host (the program calling the wasm module) and allow the wasm module to call the print function.

The Hosts Functions are a way to give super powers to the Wasm modules.

But before going further, let's create a first Wazero host program that will execute a function from a wasm module.

First contact with Wazero

Requirements:

The host program is compiled with the Go compiler

The Wasm module is compiled with the TinyGo compiler

Wasm module

Create a hello.go file. The source code of the wasm module is pretty simple:

package main

func main() {} 1️⃣

//export add 2️⃣
func add(x uint32, y uint32) uint32 {
    return x + y
}

1️⃣: main is required for TinyGo to compile to Wasm.

2️⃣: To make add function callable from host, we need to add the: export add comment above the function

To compile the wasm module, use the below command:

tinygo build -o hello.wasm \
-scheduler=none --no-debug \
-target wasi ./hello.go

Host program

Now, we will write the host program, using the. Wazero project. We need to:

Create a new WebAssembly Runtime,
Load and instantiate the WebAssembly module,
Call the Wasm function (imported by the host, exported by the module. 🖐 in fact, it not a "real import" by the host, the host will get a reference to the exported function.

Create a main.go file:

package main

import (
  "context"
  "fmt"
  "log"
  "os"

  "github.com/tetratelabs/wazero"
  "github.com/tetratelabs/wazero/wasi_snapshot_preview1"
)

func main() {
  ctx := context.Background()

  r := wazero.NewRuntimeWithConfig( 1️⃣
    ctx, wazero.NewRuntimeConfig().
    WithWasmCore2()) 2️⃣

  defer r.Close(ctx) 

  _, err := wasi_snapshot_preview1.Instantiate(ctx, r) 3️⃣
  if err != nil {
    log.Panicln(err)
  }

  helloWasm, err := os.ReadFile("./hello.wasm") 4️⃣
  if err != nil {
    log.Panicln(err)
  }

  mod, err := r.InstantiateModuleFromBinary(ctx, helloWasm) 5️⃣
  if err != nil {
    log.Panicln(err)
  }

  addWasmModuleFunction := mod.ExportedFunction("add") 6️⃣

  result, err := addWasmModuleFunction.Call(ctx, 20, 22) 7️⃣
  if err != nil {
    log.Panicln(err)
  }

  fmt.Println("result:", result[0]) 8️⃣
}

1️⃣: Create a new WebAssembly Runtime.

2️⃣: Enable WebAssembly 2.0 support, which is required for TinyGo 0.24+.

3️⃣: 🖐 This is TinyGo specific (to use WASI).

4️⃣: Load the WebAssembly module.

5️⃣: Instantiate the WebAssembly module.

6️⃣: Get a reference to the WebAssembly function: add

7️⃣: Now, we can execute the add function (result is []uint64).

8️⃣: result[0] contains the result of add(20,22), the output will be: result: 42.

You can run the host program with this command:

go run main.go 
result: 42

Ok, now, I would like to be able to print messages from the wasm module (it could be convenient for debugging purposes, for example). So, we are going to write some host functions for that.

Host functions: hostLogUint32 & hostLogString

We will write two host functions, one to print a number, the other, to print a string.

hostLogUint32: print a number to the console

We will:

On the host side:
- Write a logUint32 function.
- Export the logUint32 function to the Wasm runtime with an "export name": hostLogUint32 (you can use the same name, but prefixing the export name with host or something else help me for the readability).
On the wasm module side:
- Import the hostLogUint32 function
- Use the hostLogUint32 function to print a number to the console, from the wasm module.

Define the host function

In the main.go file (of the host program), I added the logUint32 host function :

func logUint32(value uint32) {
  fmt.Println("🤖:", value)
}

Then, after creating the wasm runtime, I exported the logUint32 function to the runtime, with hostLogUint32 as the "export name" (the name seen by the Wasm module)

_, errEnv := wasmRuntime.NewModuleBuilder("env").
  ExportFunction("hostLogUint32", logUint32).
  Instantiate(ctx, wasmRuntime)

if errEnv != nil {
  log.Panicln("env/host function(s) error:", errEnv)
}

I will provide the complete code later

Now, we can update the source code of the Wasm module.

Use the host function from the Wasm module

We will import the host function (named hostLogUint32) with this two lines:

//export hostLogUint32
func hostLogUint32(value uint32)

🖐🖐🖐 Yes, I know, it's "strange", but in this specific case, actually, //export hostLogUint32 means that the module imports the host function. And the second line allows to provide the signature of the function.

🎉 and now, we can use the host function from the Wasm module (1️⃣):

package main

func main() {}

//export hostLogUint32
func hostLogUint32(value uint32)

//export add
func add(x uint32, y uint32) uint32 {
  res := x + y
  hostLogUint32(res) 1️⃣
  return res
}

You can build the Wasm module and run the host:

tinygo build -o hello.wasm \
-scheduler=none --no-debug \
-target wasi ./hello.go

go run main.go

You'll get:

🤖: 42 # from the Wasm module
result: 42 # from the host program

You see, it's not difficult, with Wazero, to create an host function. But, now, I would like to print a "string" from the Wasm module.

hostLogString: print a string to the console

We want to call, from the Wasm module, a host function with a string parameter. But, right now, string is not a type you can use as a parameter or as a return value with a wasm function.🤔
Never mind that, we can use the linear memory, a kind of piece of memory (literally speaking) shared between the Wasm module and the Host. The linear memory is an array of bytes, and WebAssembly provides load and store instructions to manipulate bytes in this array.😳

So, more precisely, we will (from the Wasm module):

Define a string message.
Create an array of bytes from the string message.
Get a memory pointer on this array.
Get an offset from the pointer (a kind of pointer position in memory - btw, if somebody finds a better definition, I will adopt it).
Call the hostLogString host function with two parameters: the offset and the size of the buffer.
Read the memory from the host function and print the message to the console.

Fortunately, Wazero offers us some helpers to achieve this.🥰

Define the host function

In the main.go file (of the host program), I added the logString host function :

func logString(ctx context.Context, module api.Module, offset, byteCount uint32) {
  buf, ok := module.Memory().Read(ctx, offset, byteCount)
  if !ok {
    log.Panicf("Memory.Read(%d, %d) out of range", offset, byteCount)
  }
  fmt.Println("👽:", string(buf))
}

🖐 This time, we need to pass more parameters to the host function to be able to retrieve the string:

The context

The module helper

The offset (where to find the string)

The size of the buffer (or the length of the string)

Then, after creating the wasm runtime, I exported the logString function to the runtime, with hostLogString as the "export name" (the name seen by the Wasm module)

_, errEnv := wasmRuntime.NewModuleBuilder("env").
  ExportFunction("hostLogUint32", logUint32).
  ExportFunction("hostLogString", logString). //⬅️🖐
  Instantiate(ctx, wasmRuntime)

I will provide the complete code later

Now, we can update the source code of the Wasm module.

Use the new host function from the Wasm module

We will import the host function (named hostLogString) with this two lines:

//export hostLogString
func hostLogString(offset uint32, byteCount uint32)

🚀 and now, we can use the new host function from the Wasm module:

package main

import (
  "unsafe"
)

func main() {}

//export hostLogString
func hostLogString(offset uint32, byteCount uint32)

//export add
func add(x uint32, y uint32) uint32 {
  res := x + y
  offset, byteCount := strToPtr("👋 Hello from wasm")
  hostLogString(offset, byteCount)
  return res
}

// strToPtr returns a pair (`offset, byteCount`) 
// for the given string in a way
// compatible with WebAssembly numeric types.
func strToPtr(s string) (uint32, uint32) {
  buf := []byte(s)
  ptr := &buf[0]
  unsafePtr := uintptr(unsafe.Pointer(ptr))
  return uint32(unsafePtr), uint32(len(buf))
}

Build the Wasm module and run the host:

tinygo build -o hello.wasm \
-scheduler=none --no-debug \
-target wasi ./hello.go

go run main.go

You'll get:

👽: 👋 Hello from wasm # from the Wasm module
result: 42 # from the host program

You can know display complete message from the Wasm module.😃

This is the end of this long blog post. My main goal was to describe my understanding of the host functions with the Wazero project. Don't hesitate to comment and improve this blog post.

You can find the complete source code here: