VCF Automation Blog

from Stefan Schnell

C and C++ have been very popular programming languages for a long time and still are today. Many consolidated program approaches and libraries exists. With Emscripten is it possbile to transfer existing portable C / C++ projects into WebAssembly (Wasm).

Rust is a language that enforces safety programming, to code more reliable software. It can also build WebAssemblies, it is one of its compiler targets. This approach can be very interesting for new developments, because it focuses safety, control of memory layout and concurrency. It guarantees memory safety, type safety and the lack of data races.

Wasm is a binary instruction format and it is designed as a portable compilation for different targets. This means different architectures, different operating systems and different execution environments like browser, desktop or server - one for all. This makes Wasm as a very interesting approach.

Dylibso, a group of software builders, has set itself the target of making Wasm to everyone's favorite binary format. Among other interesting products, they are developing Chicory, a JVM native Wasm runtime. It allows to run Wasm programs with zero native dependencies on any Java system. So this approach can also be used with VCF Automation. This is described in this blog post.

Use C / C++ / Rust Language


First, the necessary Java archive (Jar) files must be downloaded from the MVN repository.

Store Jar and Wasm Binaries as Action

The jar files are zip packages. They are only available as a binary format. To use them in VCF Automation, they must be encoded with base64 and can then be saved as an action. The base64 encoded content can simply be copied and pasted into the action. This also applies to the Wasm file, which was built by compiling the C or Rust code.

References


C Code

Here a tiny C program with two functions, named add and hello. The add function sums two numbers. The hello function delivers a Hello World message, depending on whether a name is passed as a parameter. If a name is passed it is used, otherwise a standard text.

#include <emscripten.h>
#include <stdlib.h>
#include <string.h>

EMSCRIPTEN_KEEPALIVE 
int add(int value1, int value2) {
  int result = value1 + value2;
  return result;
}

EMSCRIPTEN_KEEPALIVE 
void hello(char* name, int len, char* result) {
  if (len == 0) {
    char* ret = "Hello World from C Language\n";
    strcpy(result, ret);
  } else {
    char* ret = "Hello ";
    strcat(ret, name);
    strcat(ret, " from C Language\n");
    strcpy(result, ret);
  }
}

Compile the C Code to WebAssembly

Now we compile the code with the following command:

@emcc wasmTest.c -o wasmTest.c.wasm --no-entry -s EXPORTED_FUNCTIONS=_malloc,_free

Emscripten eliminate functions that are not called from the compiled code. The standard C functions malloc and free are required to pass the string parameter. malloc is defined in stdlib.h an allocates memory. free is defined in stdlib.h and deallocates the space previously allocated by malloc. To make sure that the C functions are available, it must be added to the EXPORTED_FUNCTIONS.

After compilation we have a new file, wasmTest.c.wasm.

Hint: This file must also be encoded as base64 and saved as an action, as described above.

Rust Code

Here a tiny Rust program with four functions. Beside to the add and hello functions, as explained above, there are also malloc and free. These are available as equivalents for the corresponding C functions, which are exported during compilation with emscripten. The code was taken from the example, only the functions were renamed from alloc to malloc and from dealloc to free. 

use std::*;

#[no_mangle]
pub extern "C" fn malloc(len: u32) -> *mut u8 {
  let mut buf = Vec::with_capacity(len as usize);
  let ptr = buf.as_mut_ptr();
  mem::forget(buf);
  ptr
}

#[no_mangle]
pub unsafe extern "C" fn free(ptr: &mut u8, len: u32) {
  let _ = Vec::from_raw_parts(ptr, 0, len as usize);
}

#[no_mangle]
pub extern fn add(value1: i32, value2: i32) -> i32 {
  let result: i32 = value1 + value2;
  result
}

#[no_mangle]
pub extern fn hello(name: u8, len: u32, result: *mut u8) {
  let bytes = unsafe {
    slice::from_raw_parts(name as *const u8, len as usize)
  };
  let str_name = str::from_utf8(bytes).unwrap().trim();
  let mut out_text = "".to_string();
  if len == 0 {
    out_text = "Hello World from Rust Language".to_string();
  } else {
    out_text = "Hello ".to_string() + str_name + " from Rust Language";
  }
  let out = out_text.as_bytes();
  unsafe {
    std::ptr::copy(out.as_ptr().cast(), result, out.len());
  }
}

Compile the Rust Code to WebAssembly

Now we compile the code with the following command:

rustc --target wasm32-unknown-unknown -O --crate-type=cdylib wasmTest.rs -o wasmTest.rs.wasm

After compilation we have a new file, wasmTest.rs.wasm.

Hint: This file must also be encoded as base64 and saved as an action, as described above.

Comparison of C and Rust WebAssembly

Both Wasm files have the same interface, with the exception of the free function. The C function does not require the length of the allocated memory, while the Rust function expects this. But the calling of the free function in the C Wasm with an unnecessary additional parameter, in this case the memory size, does not lead to an error.
On this way, the Wasm file can be exchanged. This means that both, the C Wasm and the Rust Wasm file, can be used with the same calling program. We can see this in the following code, in which only the name of the Wasm file is exchanged and the call of the functions is identical in both cases.
This implementation shows that with a clever definition of the interface to the Wasm functions, the programming language or tool chain is irrelevant, which has built the Wasm. One calling program can be used to call Wasm functions that originate from different sources.

Execute Wasm Functions

The following JavaScript code contains in the executeWasm function several steps. After the Java archives have been loaded, the classes instantiated and the methods declared, the Wasm is instantiated and the functions are determined. At the add functions the parameters can be passed directly. At the hello function the memory, for the parameter and the return value, must be allocated first and then the parameter is set. The functions are executed and the return value is read. Finally the allocated memory is released. 

function executeWasm(self, args) {

  /**
   * @param {Array} args - Array of arguments
   * @returns string
   */

  const wasmFileName = args[0];
  const add1 = args[1];
  const add2 = args[2];

  var status = "error";

  try {

    const jarPath = "/usr/lib/vco/app-server/temp/";

    const logJarFileName = jarPath + "log-1.4.0.jar";
    const logJarFile = java.io.File(logJarFileName);

    const runtimeJarFileName = jarPath + "runtime-1.4.0.jar";
    const runtimeJarFile = java.io.File(runtimeJarFileName);

    const wasiJarFileName = jarPath + "wasi-1.4.0.jar";
    const wasiJarFile = java.io.File(wasiJarFileName);

    const wasmJarFileName = jarPath + "wasm-1.4.0.jar";
    const wasmJarFile = java.io.File(wasmJarFileName);

    const urls = java.lang.reflect.Array.newInstance(java.net.URL, 4);
    urls[0] = logJarFile.toURI().toURL();
    urls[1] = runtimeJarFile.toURI().toURL();
    urls[2] = wasiJarFile.toURI().toURL();
    urls[3] = wasmJarFile.toURI().toURL();

    const classLoader = java.net.URLClassLoader(
      urls, java.lang.ClassLoader.getSystemClassLoader()
    );
    java.lang.Thread.currentThread()
      .setContextClassLoader(classLoader);

    const cInstance = java.lang.Class.forName(
      "com.dylibso.chicory.runtime.Instance", true, classLoader
    );
    const cParser = java.lang.Class.forName(
      "com.dylibso.chicory.wasm.Parser", true, classLoader
    );
    const cWasmModule = java.lang.Class.forName(
      "com.dylibso.chicory.wasm.WasmModule", true, classLoader
    );

    const builder = cInstance.getDeclaredMethod(
      "builder", cWasmModule
    );
    const parse = cParser.getDeclaredMethod(
      "parse", java.io.File
    );

    const wasmFile = new java.io.File(wasmFileName);
    const module = parse.invoke(null, wasmFile);
    const instance = builder.invoke(null, module).build();

    const malloc = instance.export("malloc");
    const free = instance.export("free");
    const memory = instance.memory();

    // >>> Begin individual code

    const add = instance.export("add");
    const addResult = add.apply(add1, add2)[0]; // Should deliver 7

    const subtract = instance.export("subtract");
    const subtractResult = subtract.apply(5, 2)[0]; // Should deliver 3

    const hello = instance.export("hello");
    const name = "Stefan";
    const ptrName = malloc.apply(name.length)[0];
    memory.writeString(ptrName, name);
    const ptrResult = malloc.apply(128)[0];
    hello.apply(ptrName, name.length, ptrResult);
    const helloResult =
      memory.readString(ptrResult, 128).split("\0").join("").trim();
    // Should deliver Hello Stefan from C / Rust Language

    free.apply(ptrResult, 128);
    free.apply(ptrName, name.length);

    // <<< End individual code

    status = "done";

  } catch(exception) {
    return "{\"status\": " + status + 
      ", \"exception\": \"" + String(exception.message) + "\"}";
  }

  return "{\"status\": " + status + 
    ", \"add\": " + String(addResult) +
    ", \"subtract\": " + String(subtractResult) +
    ", \"hello\": \"" + helloResult + "\"}";

}

VCF Automation JavaScript Action

The action writes all the necessary files to the temporary directory. Then the Wasm function is invoked with the code and the return value is output.

/**
 * Executes a webassembly
 *
 * @param {string} in_wasmActionName - Action which contains the WASM
 * @param {string} in_wasmFileName - Name of the WASM file
 * @param {string} in_moduleName - Module which contains the action
 * @param {string} in_actionName -Action which contains the text
 *
 * @outputType {string}
 *
 * @example
 * var result = callWasm(
 *   "wasmTest_c_wasm_base64",
 *   "wasmTest.c.wasm",
 *   "de.stschnell",
 *   "executeWasm"
 * );
 * System.log(result);
 *
 * @example
 * var result = callWasm(
 *   "wasmTest_rs_wasm_base64",
 *   "wasmTest.rs.wasm",
 *   "de.stschnell",
 *   "executeWasm"
 * );
 * System.log(result);
 */

function main(
  wasmActionName,
  wasmFileName,
  moduleName,
  actionName
) {

  var output = "";

  try {

    System.getModule("de.stschnell").writeActionAsFileInTempDirectory(
      "de.stschnell",
      "log_1_4_0_jar_base64",
      "log-1.4.0.jar",
      true,
      "application/java-archive"
    );

    System.getModule("de.stschnell").writeActionAsFileInTempDirectory(
      "de.stschnell",
      "runtime_1_4_0_jar_base64",
      "runtime-1.4.0.jar",
      true,
      "application/java-archive"
    );

    System.getModule("de.stschnell").writeActionAsFileInTempDirectory(
      "de.stschnell",
      "wasi_1_4_0_jar_base64",
      "wasi-1.4.0.jar",
      true,
      "application/java-archive"
    );

    System.getModule("de.stschnell").writeActionAsFileInTempDirectory(
      "de.stschnell",
      "wasm_1_4_0_jar_base64",
      "wasm-1.4.0.jar",
      true,
      "application/java-archive"
    );

    System.getModule("de.stschnell").writeActionAsFileInTempDirectory(
      "de.stschnell",
      wasmActionName,
      wasmFileName,
      true,
      "application/wasm"
    );

    const script = System.getModule("de.stschnell").getActionAsText(
      moduleName,
      actionName
    );

    const cx = org.mozilla.javascript.Context.enter();
    cx.setLanguageVersion(org.mozilla.javascript.Context.VERSION_ES6);
    const scope = cx.initStandardObjects();

    const func = cx.compileFunction(scope, script, "script", 1, null);
    output = func.call(
      cx,
      scope,
      [ wasmFileName, 5, 2 ]
    );

  } catch (exception) {
    output = "{\"status\": " + status + 
      ", \"exception\": \"" + String(exception.message) + "\"}";
  } finally {
    cx.exit();
  }

  return output;

}

return main(
  in_wasmActionName,
  in_wasmFileName,
  in_moduleName,
  in_actionName
);

Here a part of the code and the log output in VCF Automation.

vcf automation action

Conclusion

With Emscripten it is very easy to convert C / C++ functions into Wasm. With Rust it is possible to build new developments and compile it into Wasm too. With Chicory it is very easy to use these Wasm inside VCF Automation. This approach offers possibilities to use and reuse C / C++ and Rust code seamlessly in VCF Automation. This makes it easy to combine existing code and new functionalities. And a smart definition of functions and their interfaces of a Wasm can ensure simple interchangeability, regardless of the origin of the Wasm.